StandardVAE<T>

Models & Types · AiDotNet.Diffusion.VAE

Standard Variational Autoencoder for latent diffusion models.

Inheritance: VAEModelBase<T> → StandardVAE<T>

For Beginners

The StandardVAE is like a very smart image compressor:

How it works:

  1. Encoder: Takes a 512x512x3 image and compresses it to 64x64x4 latent
  • That’s 48x compression! (786,432 values -> 16,384 values)
  • Uses multiple layers of convolutions and downsampling
  1. Decoder: Takes the 64x64x4 latent and reconstructs a 512x512x3 image
  • Uses upsampling and convolutions to expand back to full size
  • The reconstruction isn’t perfect but preserves important visual features

Why 4 latent channels?

  • The VAE learns to pack image information into 4 channels
  • Each channel captures different aspects (colors, edges, textures, etc.)
  • More channels = better quality but larger latent space

Why 8x downsampling?

  • Each side is reduced by 8 (512 -> 64)
  • This is the sweet spot between compression and quality
  • Smaller latents = faster diffusion, but potentially lower quality

How It Works

This implements a standard VAE architecture similar to Stable Diffusion’s VAE, with an encoder that compresses images to latent space and a decoder that reconstructs images from latents.

Architecture details:

  • Input: [batch, 3, H, W] RGB image normalized to [-1, 1]
  • Encoder: ResBlocks with GroupNorm, downsampling via strided conv
  • Latent: [batch, 4, H/8, W/8] with mean and variance for sampling
  • Decoder: ResBlocks with GroupNorm, upsampling via transpose conv
  • Output: [batch, 3, H, W] reconstructed image

Example

var vae = new StandardVAE<float>(inputChannels: 3, latentChannels: 4, baseChannels: 128);
var image = Tensor<float>.CreateRandom(new[] { 1, 3, 512, 512 });
var latent = vae.Encode(image);
var decoded = vae.Decode(latent);

Constructors

Constructor Summary
StandardVAE(NeuralNetworkArchitecture<T> architecture, int inputChannels, int latentChannels, int baseChannels, int[] channelMultipliers, int numResBlocksPerLevel, double? latentScaleFactor, List<ILayer<T>> encoderLayers, List<ILayer<T>> decoderLayers, ILossFunction<T> lossFunction, int? seed) Initializes a new instance of the StandardVAE class with full customization support.

Properties

Property Summary
bool SupportsSlicing { get; }
bool SupportsTiling { get; }
double LatentScaleFactor { get; }
int DownsampleFactor { get; }
int InputChannels { get; }
int LatentChannels { get; }

Methods

Method Summary
T ComputeVAELoss(Tensor<T> image, Tensor<T> reconstruction, double klWeight) Computes the VAE loss (reconstruction + KL divergence).
Tensor<T> Decode(Tensor<T> latent)
Tensor<T> DecodeFromDiffusion(Tensor<T> latent) Decodes a diffusion latent back to image space.
Tensor<T> Encode(Tensor<T> image, bool sampleMode)
Tensor<T> EncodeForDiffusion(Tensor<T> image, bool sampleMode) Encodes an image and applies latent scaling for use in diffusion.
ValueTuple<Tensor<T>, Tensor<T>> EncodeWithDistribution(Tensor<T> image)