VAEEncoder<T>

Models & Types · AiDotNet.Diffusion.VAE

Convolutional encoder for VAE that compresses images to latent space.

Inheritance: LayerBase<T> → VAEEncoder<T>

Implements: IShapeContract

For Beginners

The VAE encoder is like an intelligent image compressor.

What it does step by step:

  1. Takes a high-resolution image (e.g., 512x512x3 RGB)
  2. Initial conv: Expands channels (3 -> 128) at full resolution
  3. DownBlocks: Progressively halves resolution while increasing channels
  • Block 1: 128 channels, 512x512 -> 256x256
  • Block 2: 256 channels, 256x256 -> 128x128
  • Block 3: 512 channels, 128x128 -> 64x64
  • Block 4: 512 channels, 64x64 -> 64x64 (no downsample at end)
  1. Middle: Extra processing at the bottleneck
  2. Output: Produces mean and log-variance for 4-channel latent

The result is a 64x64x4 latent that captures the image’s essence in a compressed form suitable for diffusion.

How It Works

This implements the encoder portion of a VAE following the Stable Diffusion architecture:

  • Input convolution to initial feature channels
  • Multiple DownBlocks with ResBlocks and strided conv downsampling
  • Middle blocks with attention at the bottleneck
  • Final convolutions to produce mean and log variance for the latent distribution

Constructors

Constructor Summary
VAEEncoder(int inputChannels, int latentChannels, int baseChannels, int[] channelMults, int numResBlocks, int numGroups, int inputSpatialSize) Initializes a new instance of the VAEEncoder class.

Properties

Property Summary
bool SupportsTraining { get; }
int DownsampleFactor { get; } Gets the downsampling factor (spatial reduction from input to output).
int InputChannels { get; } Gets the number of input channels.
int LatentChannels { get; } Gets the number of latent channels.
int NamedParameterCount { get; }

Methods

Method Summary
IEnumerable<string> GetParameterNames()
IReadOnlyList<ILayer<T>> GetSubLayers()
IReadOnlyList<OutputAxisContract> OutputAxesFor(int inputRank)
Tensor<T> EncodeAndSample(Tensor<T> input, int? seed) Encodes an image and samples from the latent distribution.
ValueTask<Tensor<T>> ForwardAsync(Tensor<T> input, CancellationToken cancellationToken) Async overload of Forward.
ValueTuple<Tensor<T>, Tensor<T>> EncodeWithDistribution(Tensor<T> input) Encodes and returns mean and log variance separately.
WeightLoadResult LoadWeights(Dictionary<string, Tensor<T>> weights, Func<string, string> mapping, bool strict)
WeightLoadValidation ValidateWeights(IEnumerable<string> weightNames, Func<string, string> mapping)
bool SetParameter(string name, Tensor<T> value)
bool TryGetParameter(string name, Tensor<T> tensor)
int[] GetParameterShape(string name)
void InvalidateCompiledPlans() Bumps the structure-version counter so the next call to Forward drops any plan compiled against an older graph topology.
void ResetState() Resets the internal state of the encoder.
void UpdateParameters(T learningRate) Updates all learnable parameters using gradient descent.