VAEEncoder<T>
Models & Types · AiDotNet.Diffusion.VAE
Convolutional encoder for VAE that compresses images to latent space.
Inheritance: LayerBase<T> → VAEEncoder<T>
Implements: IShapeContract
For Beginners
The VAE encoder is like an intelligent image compressor.
What it does step by step:
- Takes a high-resolution image (e.g., 512x512x3 RGB)
- Initial conv: Expands channels (3 -> 128) at full resolution
- DownBlocks: Progressively halves resolution while increasing channels
- Block 1: 128 channels, 512x512 -> 256x256
- Block 2: 256 channels, 256x256 -> 128x128
- Block 3: 512 channels, 128x128 -> 64x64
- Block 4: 512 channels, 64x64 -> 64x64 (no downsample at end)
- Middle: Extra processing at the bottleneck
- Output: Produces mean and log-variance for 4-channel latent
The result is a 64x64x4 latent that captures the image’s essence in a compressed form suitable for diffusion.
How It Works
This implements the encoder portion of a VAE following the Stable Diffusion architecture:
- Input convolution to initial feature channels
- Multiple DownBlocks with ResBlocks and strided conv downsampling
- Middle blocks with attention at the bottleneck
- Final convolutions to produce mean and log variance for the latent distribution
Constructors
| Constructor | Summary |
|---|---|
VAEEncoder(int inputChannels, int latentChannels, int baseChannels, int[] channelMults, int numResBlocks, int numGroups, int inputSpatialSize) |
Initializes a new instance of the VAEEncoder class. |
Properties
| Property | Summary |
|---|---|
bool SupportsTraining { get; } |
|
int DownsampleFactor { get; } |
Gets the downsampling factor (spatial reduction from input to output). |
int InputChannels { get; } |
Gets the number of input channels. |
int LatentChannels { get; } |
Gets the number of latent channels. |
int NamedParameterCount { get; } |
Methods
| Method | Summary |
|---|---|
IEnumerable<string> GetParameterNames() |
|
IReadOnlyList<ILayer<T>> GetSubLayers() |
|
IReadOnlyList<OutputAxisContract> OutputAxesFor(int inputRank) |
|
Tensor<T> EncodeAndSample(Tensor<T> input, int? seed) |
Encodes an image and samples from the latent distribution. |
ValueTask<Tensor<T>> ForwardAsync(Tensor<T> input, CancellationToken cancellationToken) |
Async overload of Forward. |
ValueTuple<Tensor<T>, Tensor<T>> EncodeWithDistribution(Tensor<T> input) |
Encodes and returns mean and log variance separately. |
WeightLoadResult LoadWeights(Dictionary<string, Tensor<T>> weights, Func<string, string> mapping, bool strict) |
|
WeightLoadValidation ValidateWeights(IEnumerable<string> weightNames, Func<string, string> mapping) |
|
bool SetParameter(string name, Tensor<T> value) |
|
bool TryGetParameter(string name, Tensor<T> tensor) |
|
int[] GetParameterShape(string name) |
|
void InvalidateCompiledPlans() |
Bumps the structure-version counter so the next call to Forward drops any plan compiled against an older graph topology. |
void ResetState() |
Resets the internal state of the encoder. |
void UpdateParameters(T learningRate) |
Updates all learnable parameters using gradient descent. |