IResidualVectorQuantizer<T>

Interfaces · AiDotNet.Audio.Codecs

A residual vector quantizer as the neural audio codecs use it: [1, dim, T] in, the sum of the code vectors of the active codebooks out, with codes per codebook.

Known implementations: FactorizedVectorQuantizerLayer<T>, ResidualVectorQuantizerLayer<T>

Properties

Property Summary
Tensor<T> CommitmentLoss { get; } The commitment loss of the last training forward (a tape-connected scalar), or null.
bool CodebooksInitialized { get; } Whether every codebook has been fitted (by its data-dependent initialization or a load); a quantizer without one is always fitted.
int ActiveQuantizers { get; set; } The number of codebooks the forward uses (all when 0).

Methods

Method Summary
Tensor<T> Decode(int[] codes) The quantized latent [1, dim, T] of codes [n_q, T].
Tensor<T> Forward(Tensor<T> latent) The quantized latent of the last forward, straight-through to its input.
int[] Encode(Tensor<T> latent, int? quantizers) The codes [n_q, T] of a latent [1, dim, T].
void InitializeCodebooks(Tensor<T> latent) Fits every codebook that has never been fitted from a latent [1, dim, T], as the first training batch does, without updating fitted ones.