ConsistencyModel<T>

Models & Types · AiDotNet.Diffusion.FastGeneration

Consistency Model for single-step or few-step image generation.

Inheritance: DiffusionModelBase<T> → LatentDiffusionModelBase<T> → ConsistencyModel<T>

For Beginners

Traditional diffusion models are like walking down a path one step at a time - they need many steps to go from noise to a clear image. Consistency Models are like teleportation - they can jump directly from any point on the path to the destination (clean image).

Key advantages:

  • Single-step generation possible (fastest mode)
  • Progressive refinement: 1-step, 2-step, 4-step, etc.
  • Quality improves with more steps but plateaus quickly
  • Same or better quality as DDPM with 1000x fewer steps

How it works:

  1. The model learns that all points on a denoising path should map to the same clean image
  2. This “consistency” property allows direct prediction from any noise level
  3. The model can self-refine by treating its output as a new starting point

Use cases:

  • Real-time image generation
  • Interactive applications
  • Mobile/edge deployment
  • Batch processing at scale

How It Works

Consistency Models can generate high-quality images in a single step by learning to map any point on a probability flow ODE trajectory directly to the trajectory’s origin (the clean data). This enables extremely fast generation compared to traditional diffusion models that require 20-50+ steps.

Technical details:

  • Based on probability flow ODEs (deterministic diffusion)
  • Two training methods: distillation from pretrained diffusion, direct training
  • Uses boundary condition f(x, eps) = x (identity at minimal noise)
  • Supports multistep sampling for quality/speed tradeoff

Reference: Song et al., “Consistency Models”, ICML 2023

Example

// Create a Consistency Model
var model = new ConsistencyModel<float>();
            
// Single-step generation (fastest)
var image1 = model.GenerateFromText(
    prompt: "A beautiful sunset over mountains",
    numInferenceSteps: 1);
            
// Two-step generation (better quality)
var image2 = model.GenerateFromText(
    prompt: "A beautiful sunset over mountains",
    numInferenceSteps: 2);
            
// Four-step generation (highest quality)
var image4 = model.GenerateFromText(
    prompt: "A beautiful sunset over mountains",
    numInferenceSteps: 4);
            
// Generate with progressive refinement
var refined = model.GenerateWithProgressiveRefinement(
    prompt: "Detailed landscape painting",
    maxSteps: 4,
    returnIntermediates: true);

Constructors

Constructor Summary
ConsistencyModel(NeuralNetworkArchitecture<T> architecture, DiffusionModelOptions<T> options, INoiseScheduler<T> scheduler, UNetNoisePredictor<T> noisePredictor, StandardVAE<T> vae, IConditioningModule<T> conditioner, int numTrainSteps, double sigmaMin, double sigmaMax, double rho, bool isDistilled, int? seed) Initializes a new instance of ConsistencyModel with default parameters.

Properties

Property Summary
IConditioningModule<T> Conditioner { get; }
INoisePredictor<T> NoisePredictor { get; }
IVAEModel<T> VAE { get; }
bool IsDistilled { get; } Gets whether this model was trained via distillation.
double SigmaMax { get; } Gets the maximum sigma value used by this model.
double SigmaMin { get; } Gets the minimum sigma value used by this model.
int LatentChannels { get; }

Methods

Method Summary
List<Tensor<T>> GenerateWithProgressiveRefinement(string prompt, string negativePrompt, int width, int height, int maxSteps, bool returnIntermediates, double? guidanceScale, int? seed) Generates images with progressive refinement, optionally returning intermediates.
ModelMetadata<T> GetModelMetadata()
Tensor<T> ConsistencyFunction(Tensor<T> x, T sigma, Tensor<T> conditioning) Applies the consistency function to map from any noise level to clean data.
Tensor<T> GenerateFromText(string prompt, string negativePrompt, int width, int height, int numInferenceSteps, double? guidanceScale, int? seed) Generates an image using consistency sampling.