ConsistencyModel<T>
Models & Types · AiDotNet.Diffusion.FastGeneration
Consistency Model for single-step or few-step image generation.
Inheritance: DiffusionModelBase<T> → LatentDiffusionModelBase<T> → ConsistencyModel<T>
For Beginners
Traditional diffusion models are like walking down a path one step at a time - they need many steps to go from noise to a clear image. Consistency Models are like teleportation - they can jump directly from any point on the path to the destination (clean image).
Key advantages:
- Single-step generation possible (fastest mode)
- Progressive refinement: 1-step, 2-step, 4-step, etc.
- Quality improves with more steps but plateaus quickly
- Same or better quality as DDPM with 1000x fewer steps
How it works:
- The model learns that all points on a denoising path should map to the same clean image
- This “consistency” property allows direct prediction from any noise level
- The model can self-refine by treating its output as a new starting point
Use cases:
- Real-time image generation
- Interactive applications
- Mobile/edge deployment
- Batch processing at scale
How It Works
Consistency Models can generate high-quality images in a single step by learning to map any point on a probability flow ODE trajectory directly to the trajectory’s origin (the clean data). This enables extremely fast generation compared to traditional diffusion models that require 20-50+ steps.
Technical details:
- Based on probability flow ODEs (deterministic diffusion)
- Two training methods: distillation from pretrained diffusion, direct training
- Uses boundary condition f(x, eps) = x (identity at minimal noise)
- Supports multistep sampling for quality/speed tradeoff
Reference: Song et al., “Consistency Models”, ICML 2023
Example
// Create a Consistency Model
var model = new ConsistencyModel<float>();
// Single-step generation (fastest)
var image1 = model.GenerateFromText(
prompt: "A beautiful sunset over mountains",
numInferenceSteps: 1);
// Two-step generation (better quality)
var image2 = model.GenerateFromText(
prompt: "A beautiful sunset over mountains",
numInferenceSteps: 2);
// Four-step generation (highest quality)
var image4 = model.GenerateFromText(
prompt: "A beautiful sunset over mountains",
numInferenceSteps: 4);
// Generate with progressive refinement
var refined = model.GenerateWithProgressiveRefinement(
prompt: "Detailed landscape painting",
maxSteps: 4,
returnIntermediates: true);
Constructors
| Constructor | Summary |
|---|---|
ConsistencyModel(NeuralNetworkArchitecture<T> architecture, DiffusionModelOptions<T> options, INoiseScheduler<T> scheduler, UNetNoisePredictor<T> noisePredictor, StandardVAE<T> vae, IConditioningModule<T> conditioner, int numTrainSteps, double sigmaMin, double sigmaMax, double rho, bool isDistilled, int? seed) |
Initializes a new instance of ConsistencyModel with default parameters. |
Properties
| Property | Summary |
|---|---|
IConditioningModule<T> Conditioner { get; } |
|
INoisePredictor<T> NoisePredictor { get; } |
|
IVAEModel<T> VAE { get; } |
|
bool IsDistilled { get; } |
Gets whether this model was trained via distillation. |
double SigmaMax { get; } |
Gets the maximum sigma value used by this model. |
double SigmaMin { get; } |
Gets the minimum sigma value used by this model. |
int LatentChannels { get; } |
Methods
| Method | Summary |
|---|---|
List<Tensor<T>> GenerateWithProgressiveRefinement(string prompt, string negativePrompt, int width, int height, int maxSteps, bool returnIntermediates, double? guidanceScale, int? seed) |
Generates images with progressive refinement, optionally returning intermediates. |
ModelMetadata<T> GetModelMetadata() |
|
Tensor<T> ConsistencyFunction(Tensor<T> x, T sigma, Tensor<T> conditioning) |
Applies the consistency function to map from any noise level to clean data. |
Tensor<T> GenerateFromText(string prompt, string negativePrompt, int width, int height, int numInferenceSteps, double? guidanceScale, int? seed) |
Generates an image using consistency sampling. |