ILatentDiffusionModel<T>
Interfaces · AiDotNet.Interfaces
Interface for latent diffusion models that operate in a compressed latent space.
Implements: IDiffusionModel<T>
Known implementations: LatentDiffusionModelBase<T>
For Beginners
Latent diffusion combines the power of diffusion models with the efficiency of autoencoders.
How it works:
- A VAE compresses images (512x512) into small latents (64x64)
- Diffusion happens in this compressed space (much faster!)
- The VAE decompresses the result back to a full image
Benefits:
- Training is ~50x faster than pixel-space diffusion
- Generation is ~50x faster
- Quality remains very high
- Enables practical high-resolution generation
Key components:
- VAE: Compresses and decompresses images
- Noise Predictor (U-Net/DiT): Predicts noise in latent space
- Scheduler: Controls the denoising process
- Conditioner: Encodes text/images for guided generation
How It Works
Latent diffusion models are a highly efficient variant of diffusion models that perform the denoising process in a compressed latent space rather than pixel space. This is the architecture behind Stable Diffusion and many other state-of-the-art generative models.
This interface extends IDiffusionModel with latent-space specific operations.
Properties
| Property | Summary |
|---|---|
IConditioningModule<T> Conditioner { get; } |
Gets the conditioning module (optional, for conditioned generation). |
INoisePredictor<T> NoisePredictor { get; } |
Gets the noise predictor model (U-Net, DiT, etc.). |
IVAEModel<T> VAE { get; } |
Gets the VAE model used for encoding and decoding. |
bool SupportsInpainting { get; } |
Gets whether this model supports inpainting. |
bool SupportsNegativePrompt { get; } |
Gets whether this model supports negative prompts. |
double GuidanceScale { get; } |
Gets the default guidance scale for classifier-free guidance. |
int LatentChannels { get; } |
Gets the number of latent channels. |
Methods
| Method | Summary |
|---|---|
Tensor<T> DecodeFromLatent(Tensor<T> latent) |
Decodes a latent representation back to an image. |
Tensor<T> EncodeToLatent(Tensor<T> image, bool sampleMode) |
Encodes an image into latent space. |
Tensor<T> GenerateFromText(string prompt, string negativePrompt, int width, int height, int numInferenceSteps, double? guidanceScale, int? seed) |
Generates images from text prompts using classifier-free guidance. |
Tensor<T> ImageToImage(Tensor<T> inputImage, string prompt, string negativePrompt, double strength, int numInferenceSteps, double? guidanceScale, int? seed) |
Performs image-to-image generation (style transfer, editing). |
Tensor<T> Inpaint(Tensor<T> inputImage, Tensor<T> mask, string prompt, string negativePrompt, int numInferenceSteps, double? guidanceScale, int? seed) |
Performs inpainting (filling in masked regions). |
void SetGuidanceScale(double scale) |
Sets the guidance scale for classifier-free guidance. |