FactorizedVectorQuantizerLayer<T>
Layers · AiDotNet.Audio.Codecs
DAC’s residual vector quantizer with factorized, L2-normalized codes (Kumar et al. 2023, §3.2 and App. A; reference
dac/nn/quantize.py): each codebook projects its residual to a low-dimensional space (W_in, a weight-normalized
1×1 convolution), looks up the nearest code by cosine similarity (both sides L2-normalized), and projects the code
back (W_out); the codebooks are learned by gradient with the VQ-VAE codebook and commitment losses and the
straight-through estimator — no EMA, k-means initialization or restarts.
Inheritance: LayerBase<T> → FactorizedVectorQuantizerLayer<T>
Implements: IResidualVectorQuantizer<T>, IShapeContract
How It Works
App. A states both losses on the L2-normalized vectors:
L_VQ = ‖sg[ℓ2(z_proj)] − ℓ2(e_k)‖² + β‖ℓ2(z_proj) − sg[ℓ2(e_k)]‖²; the reference code takes them on the raw
projections and codes (LossesOnRawVectors). Each is the mean squared error over the projected dimensions
and frames, summed over the codebooks used.
Constructors
| Constructor | Summary |
|---|---|
FactorizedVectorQuantizerLayer(int dim, int quantizers, int bins, int codeDim, bool lossesOnRawVectors, bool useL1Loss) |
Creates the quantizer. |
Properties
| Property | Summary |
|---|---|
Tensor<T> CodebookLoss { get; set; } |
The codebook loss of the last training forward, summed over the codebooks used. |
Tensor<T> CommitmentLoss { get; set; } |
|
bool CodebooksInitialized { get; } |
|
bool LossesOnRawVectors { get; } |
Whether the losses compare raw vectors (the reference code) rather than L2-normalized ones. |
bool SupportsTraining { get; } |
|
int ActiveQuantizers { get; set; } |
|
int[] LastCodes { get; } |
The codes [n_q, T] of the last forward (read from its one-hot codes when first asked for). |
Methods
| Method | Summary |
|---|---|
IReadOnlyList<ILayer<T>> GetSubLayers() |
|
IReadOnlyList<OutputAxisContract> OutputAxesFor(int inputRank) |
Derived from this type’s [TensorLayout] declarations. |
Tensor<T> Codebook(int q) |
Codebook q``[bins, codeDim]. |
Tensor<T> Decode(int[] codes) |
|
int[] Encode(Tensor<T> latent, int? quantizers) |
|
void InitializeCodebooks(Tensor<T> latent) |
|
void ResetState() |