MultinomialDiffusion<T>

Models & Types · AiDotNet.NeuralNetworks.SyntheticData

Implements the multinomial diffusion process for categorical features in TabDDPM.

For Beginners

This handles the “category” part of the diffusion process.

Forward (corrupting categories):

  • Start with a real category (e.g., color = “red”)
  • At each step, there’s a small chance the category “flips” to a random one
  • After many steps, the category is essentially random (equally likely to be any color)

Reverse (recovering categories):

  • Start with a random category
  • The model predicts the probability of each possible category
  • Sample from those probabilities to get a cleaner category
  • After many steps, you get a realistic category assignment

The noise schedule controls how quickly categories become randomized.

How It Works

Multinomial diffusion operates on categorical (one-hot) features:

  • Forward process: Gradually mixes the true category distribution toward uniform

q(x_t | x_0) = (1 - beta_t) * x_{t-1} + beta_t * (1/K) where K is the number of categories

  • Training: The model predicts log-probabilities of the original categories
  • Loss: KL divergence between predicted posterior and true posterior
  • Reverse process: Sample categories from the predicted probability distribution

Constructors

Constructor Summary
MultinomialDiffusion(int numTimesteps, double betaStart, double betaEnd, Random random) Initializes a new multinomial diffusion process.

Properties

Property Summary
int NumTimesteps { get; } Gets the number of diffusion timesteps.

Methods

Method Summary
T ComputeLoss(Vector<T> predictedLogProbs, Vector<T> trueOneHot, Vector<T> noisyProbs, int t) Computes the KL divergence loss between predicted and true posterior.
Vector<T> AddNoise(Vector<T> x0, int t) Adds categorical noise to a one-hot vector at timestep t (forward process).
Vector<T> ComputeLossGradient(Vector<T> predictedLogProbs, Vector<T> trueOneHot) Computes the gradient of the auxiliary x_0 cross-entropy with respect to predicted log-probabilities.
Vector<T> ComputeLossGradient(Vector<T> predictedLogProbs, Vector<T> trueOneHot, Vector<T> noisyProbs, int t) Computes the gradient of ComputeLoss with respect to predicted log-probabilities.
Vector<T> DenoisingStep(Vector<T> noisyProbs, Vector<T> predictedLogProbs, int t) Performs one reverse diffusion step for categorical features.
int SampleTimestep() Samples a random timestep uniformly from [0, numTimesteps).