ConsistencyTrainingTrainer<T>
Models & Types · AiDotNet.Diffusion.Distillation
Trainer for consistency training from scratch without a pretrained teacher model.
For Beginners
While consistency distillation needs a “teacher” model to learn from, consistency training learns on its own. It discovers that different amounts of noise added to the same image should all map back to the same clean image. This is like learning to recognize a face from photos taken at different exposure levels — they’re all the same face.
How It Works
Unlike consistency distillation (which requires a pretrained teacher), consistency training learns the consistency function directly from data using a self-consistency loss. The model learns that mapping any point along the same ODE trajectory should produce the same output, without needing teacher guidance. Uses a progressive schedule that increases discretization steps during training.
Reference: Song et al., “Improved Techniques for Training Consistency Models”, ICML 2024 (iCT); Song and Dhariwal, “Consistency Models”, ICML 2023
Constructors
| Constructor | Summary |
|---|---|
ConsistencyTrainingTrainer(double emaDecay, int initialDiscretizationSteps, int finalDiscretizationSteps, double sigmaMin, double sigmaMax) |
Initializes a new consistency training trainer. |
Properties
| Property | Summary |
|---|---|
int CurrentStep { get; } |
Gets the current training step. |
int InitialDiscretizationSteps { get; } |
Gets the initial discretization step count. |
Methods
| Method | Summary |
|---|---|
T ComputeConsistencyLoss(Vector<T> modelOutputAtT, Vector<T> targetOutputAtTPlusOne) |
Computes the self-consistency loss for a training batch. |
T GetSigma(int index, int numSteps) |
Gets the sigma (noise level) for a given discretization index. |
double GetAdaptiveEMADecay(int totalTrainingSteps) |
Computes the adaptive EMA decay rate based on current discretization. |
int GetCurrentDiscretizationSteps(int totalTrainingSteps) |
Gets the current number of discretization steps based on training progress. |
void Step() |
Advances the training step counter. |