VAEModelBase<T>
Base Classes · AiDotNet.Diffusion.VAE
Base class for Variational Autoencoder (VAE) models used in latent diffusion.
Implements: IModelShape, IParameterChunkSource<T>, IParameterManifestProvider, IParameterSurfaceLifecycle, IVAEModel<T>
Known derived types: AudioVAE<T>, AutoencoderKL<T>, Causal3DVAE<T>, DeepCompressionVAE<T>, EQVAEModel<T>, ImprovedVideoVAE<T>, LiteVAEModel<T>, SDXLVAEModel<T>, StandardVAE<T>, TemporalInterpolationVAE<T>, TemporalVAE<T>
For Beginners
This is the foundation for all VAE models in the library. VAEs compress images to a small latent representation and decompress them back. They are essential for efficient latent diffusion models like Stable Diffusion.
How It Works
This abstract base class provides common functionality for all VAE implementations, including encoding, decoding, sampling, and latent scaling operations.
Properties
| Property | Summary |
|---|---|
ILossFunction<T> DefaultLossFunction { get; } |
|
ParameterLayoutSnapshot ParameterLayout { get; } |
|
bool SupportsParameterInitialization { get; } |
|
bool SupportsSlicing { get; } |
|
bool SupportsTiling { get; } |
|
double LatentScaleFactor { get; } |
|
double TrainingLearningRate { get; set; } |
Gets or sets the Adam learning rate Train uses, read when its optimizer is first built. |
int DownsampleFactor { get; } |
|
int InputChannels { get; } |
|
int LatentChannels { get; } |
|
long ParameterCount { get; } |
Methods
| Method | Summary |
|---|---|
Dictionary<string, T> GetFeatureImportance() |
|
DynamicShapeInfo GetDynamicShapeInfo() |
|
IEnumerable<ParameterChunk<T>> GetParameterStateChunks() |
Exposes canonical registered state for device-side checkpoints. |
IEnumerable<Tensor<T>> GetParameterChunks() |
Streams the VAE’s registered parameter state in canonical stable-ID order without materialising a flat aggregate, mirroring PyTorch’s nn.Module.parameters() generator pattern. |
IEnumerable<int> GetActiveFeatureIndices() |
|
IFullModel<T, Tensor<T>, Tensor<T>> DeepCopy() |
|
IFullModel<T, Tensor<T>, Tensor<T>> WithParameters(Vector<T> parameters) |
|
IVAEModel<T> Clone() |
Creates a deep copy of the VAE model. |
ModelMetadata<T> GetModelMetadata() |
|
Tensor<T> Decode(Tensor<T> latent) |
|
Tensor<T> Encode(Tensor<T> image, bool sampleMode) |
|
Tensor<T> Predict(Tensor<T> input) |
|
Tensor<T> Sample(Tensor<T> mean, Tensor<T> logVariance, int? seed) |
|
Tensor<T> ScaleLatent(Tensor<T> latent) |
|
Tensor<T> UnscaleLatent(Tensor<T> latent) |
|
ValueTuple<Tensor<T>, Tensor<T>> EncodeWithDistribution(Tensor<T> image) |
|
Vector<T> ComputeGradients(Tensor<T> input, Tensor<T> target, ILossFunction<T> lossFunction) |
|
Vector<T> GetParameters() |
|
Vector<T> SanitizeParameters(Vector<T> parameters) |
|
bool IsFeatureUsed(int featureIndex) |
|
byte[] Serialize() |
|
int[] GetInputShape() |
|
int[] GetOutputShape() |
|
void ApplyGradients(Vector<T> gradients, T learningRate) |
|
void Deserialize(byte[] data) |
|
void Dispose() |
|
void LoadModel(string filePath) |
|
void LoadState(Stream stream) |
|
void SaveModel(string filePath) |
|
void SaveState(Stream stream) |
|
void SetActiveFeatureIndices(IEnumerable<int> featureIndices) |
|
void SetParameterChunks(IEnumerable<Tensor<T>> chunks) |
Streaming counterpart to SetParameters: assigns the VAE’s weights from per-tensor chunks supplied in the SAME order GetParameterChunks yields them. |
void SetParameters(Vector<T> parameters) |
|
void SetSlicingEnabled(bool enabled) |
|
void SetTilingEnabled(bool enabled) |
|
void Train(Tensor<T> input, Tensor<T> expectedOutput) |