StandardVAE<T>
Models & Types · AiDotNet.Diffusion.VAE
Standard Variational Autoencoder for latent diffusion models.
Inheritance: VAEModelBase<T> → StandardVAE<T>
For Beginners
The StandardVAE is like a very smart image compressor:
How it works:
- Encoder: Takes a 512x512x3 image and compresses it to 64x64x4 latent
- That’s 48x compression! (786,432 values -> 16,384 values)
- Uses multiple layers of convolutions and downsampling
- Decoder: Takes the 64x64x4 latent and reconstructs a 512x512x3 image
- Uses upsampling and convolutions to expand back to full size
- The reconstruction isn’t perfect but preserves important visual features
Why 4 latent channels?
- The VAE learns to pack image information into 4 channels
- Each channel captures different aspects (colors, edges, textures, etc.)
- More channels = better quality but larger latent space
Why 8x downsampling?
- Each side is reduced by 8 (512 -> 64)
- This is the sweet spot between compression and quality
- Smaller latents = faster diffusion, but potentially lower quality
How It Works
This implements a standard VAE architecture similar to Stable Diffusion’s VAE, with an encoder that compresses images to latent space and a decoder that reconstructs images from latents.
Architecture details:
- Input: [batch, 3, H, W] RGB image normalized to [-1, 1]
- Encoder: ResBlocks with GroupNorm, downsampling via strided conv
- Latent: [batch, 4, H/8, W/8] with mean and variance for sampling
- Decoder: ResBlocks with GroupNorm, upsampling via transpose conv
- Output: [batch, 3, H, W] reconstructed image
Example
var vae = new StandardVAE<float>(inputChannels: 3, latentChannels: 4, baseChannels: 128);
var image = Tensor<float>.CreateRandom(new[] { 1, 3, 512, 512 });
var latent = vae.Encode(image);
var decoded = vae.Decode(latent);
Constructors
| Constructor | Summary |
|---|---|
StandardVAE(NeuralNetworkArchitecture<T> architecture, int inputChannels, int latentChannels, int baseChannels, int[] channelMultipliers, int numResBlocksPerLevel, double? latentScaleFactor, List<ILayer<T>> encoderLayers, List<ILayer<T>> decoderLayers, ILossFunction<T> lossFunction, int? seed) |
Initializes a new instance of the StandardVAE class with full customization support. |
Properties
| Property | Summary |
|---|---|
bool SupportsSlicing { get; } |
|
bool SupportsTiling { get; } |
|
double LatentScaleFactor { get; } |
|
int DownsampleFactor { get; } |
|
int InputChannels { get; } |
|
int LatentChannels { get; } |
Methods
| Method | Summary |
|---|---|
T ComputeVAELoss(Tensor<T> image, Tensor<T> reconstruction, double klWeight) |
Computes the VAE loss (reconstruction + KL divergence). |
Tensor<T> Decode(Tensor<T> latent) |
|
Tensor<T> DecodeFromDiffusion(Tensor<T> latent) |
Decodes a diffusion latent back to image space. |
Tensor<T> Encode(Tensor<T> image, bool sampleMode) |
|
Tensor<T> EncodeForDiffusion(Tensor<T> image, bool sampleMode) |
Encodes an image and applies latent scaling for use in diffusion. |
ValueTuple<Tensor<T>, Tensor<T>> EncodeWithDistribution(Tensor<T> image) |