VAEModelBase<T>

Base Classes · AiDotNet.Diffusion.VAE

Base class for Variational Autoencoder (VAE) models used in latent diffusion.

Implements: IModelShape, IParameterChunkSource<T>, IParameterManifestProvider, IParameterSurfaceLifecycle, IVAEModel<T>

Known derived types: AudioVAE<T>, AutoencoderKL<T>, Causal3DVAE<T>, DeepCompressionVAE<T>, EQVAEModel<T>, ImprovedVideoVAE<T>, LiteVAEModel<T>, SDXLVAEModel<T>, StandardVAE<T>, TemporalInterpolationVAE<T>, TemporalVAE<T>

For Beginners

This is the foundation for all VAE models in the library. VAEs compress images to a small latent representation and decompress them back. They are essential for efficient latent diffusion models like Stable Diffusion.

How It Works

This abstract base class provides common functionality for all VAE implementations, including encoding, decoding, sampling, and latent scaling operations.

Properties

Property Summary
ILossFunction<T> DefaultLossFunction { get; }
ParameterLayoutSnapshot ParameterLayout { get; }
bool SupportsParameterInitialization { get; }
bool SupportsSlicing { get; }
bool SupportsTiling { get; }
double LatentScaleFactor { get; }
double TrainingLearningRate { get; set; } Gets or sets the Adam learning rate Train uses, read when its optimizer is first built.
int DownsampleFactor { get; }
int InputChannels { get; }
int LatentChannels { get; }
long ParameterCount { get; }

Methods

Method Summary
Dictionary<string, T> GetFeatureImportance()
DynamicShapeInfo GetDynamicShapeInfo()
IEnumerable<ParameterChunk<T>> GetParameterStateChunks() Exposes canonical registered state for device-side checkpoints.
IEnumerable<Tensor<T>> GetParameterChunks() Streams the VAE’s registered parameter state in canonical stable-ID order without materialising a flat aggregate, mirroring PyTorch’s nn.Module.parameters() generator pattern.
IEnumerable<int> GetActiveFeatureIndices()
IFullModel<T, Tensor<T>, Tensor<T>> DeepCopy()
IFullModel<T, Tensor<T>, Tensor<T>> WithParameters(Vector<T> parameters)
IVAEModel<T> Clone() Creates a deep copy of the VAE model.
ModelMetadata<T> GetModelMetadata()
Tensor<T> Decode(Tensor<T> latent)
Tensor<T> Encode(Tensor<T> image, bool sampleMode)
Tensor<T> Predict(Tensor<T> input)
Tensor<T> Sample(Tensor<T> mean, Tensor<T> logVariance, int? seed)
Tensor<T> ScaleLatent(Tensor<T> latent)
Tensor<T> UnscaleLatent(Tensor<T> latent)
ValueTuple<Tensor<T>, Tensor<T>> EncodeWithDistribution(Tensor<T> image)
Vector<T> ComputeGradients(Tensor<T> input, Tensor<T> target, ILossFunction<T> lossFunction)
Vector<T> GetParameters()
Vector<T> SanitizeParameters(Vector<T> parameters)
bool IsFeatureUsed(int featureIndex)
byte[] Serialize()
int[] GetInputShape()
int[] GetOutputShape()
void ApplyGradients(Vector<T> gradients, T learningRate)
void Deserialize(byte[] data)
void Dispose()
void LoadModel(string filePath)
void LoadState(Stream stream)
void SaveModel(string filePath)
void SaveState(Stream stream)
void SetActiveFeatureIndices(IEnumerable<int> featureIndices)
void SetParameterChunks(IEnumerable<Tensor<T>> chunks) Streaming counterpart to SetParameters: assigns the VAE’s weights from per-tensor chunks supplied in the SAME order GetParameterChunks yields them.
void SetParameters(Vector<T> parameters)
void SetSlicingEnabled(bool enabled)
void SetTilingEnabled(bool enabled)
void Train(Tensor<T> input, Tensor<T> expectedOutput)