DeepCompressionVAE<T>

Models & Types · AiDotNet.Diffusion.VAE

Deep Compression Autoencoder (DC-AE) for extremely high spatial compression.

Inheritance: VAEModelBase<T> → DeepCompressionVAE<T>

Implements: IGeneratedParameterRegistrar<T>

For Beginners

Standard VAEs compress images 8x spatially (512x512 → 64x64). DC-AE compresses 32x-128x (512x512 → 16x16 or even 4x4), making diffusion dramatically faster because the model works on much smaller latent tensors. The two-stage training ensures image quality doesn’t suffer despite the extreme compression.

How It Works

DC-AE achieves 32x-128x spatial compression (vs standard 8x) by combining residual autoencoding with a decoupled two-stage training: first train a standard AE, then add a lightweight latent adapter to achieve extreme compression while preserving quality.

Reference: Chen et al., “Deep Compression Autoencoder for Efficient High-Resolution Diffusion Models”, NeurIPS 2024

Example

var vae = new DeepCompressionVAE<float>(inputChannels: 3, latentChannels: 64, downsampleFactor: 32);
var image = Tensor<float>.CreateRandom(new[] { 1, 3, 512, 512 });
var latent = vae.Encode(image); // 16x16x64

Constructors

Constructor Summary
DeepCompressionVAE(int inputChannels, int latentChannels, int downsampleFactor, int baseChannels, ILossFunction<T> lossFunction, int? seed) Initializes a new Deep Compression Autoencoder.

Properties

Property Summary
bool SupportsTiling { get; }
double LatentScaleFactor { get; }
int DownsampleFactor { get; }
int InputChannels { get; }
int LatentChannels { get; }

Methods

Method Summary
Tensor<T> Decode(Tensor<T> latent)
Tensor<T> Encode(Tensor<T> image, bool sampleMode)
ValueTuple<Tensor<T>, Tensor<T>> EncodeWithDistribution(Tensor<T> image)