DAC<T>
Models & Types · AiDotNet.Audio.Effects
DAC, the Descript Audio Codec: a Snake-activated convolutional encoder–decoder with a residual vector quantizer of factorized, L2-normalized codes, trained with quantizer dropout against multi-period and multi-band multi-scale STFT discriminators and a multi-scale mel loss.
Inheritance: NeuralNetworkBase<T> → AudioNeuralNetworkBase<T> → NeuralAudioCodecBase<T> → DAC<T>
For Beginners
DAC compresses any kind of audio — speech, music, sound effects — into about 86 frames of codes per second and reconstructs it with very little loss; using fewer codebooks lowers the bitrate.
How It Works
Reference: “High-Fidelity Audio Compression with Improved RVQGAN” (Kumar et al., 2023), with descriptinc/descript-audio-codec for what the paper leaves unstated.
The encoder and decoder (§3.1, §4.3) are SoundStream’s fully convolutional design with Snake activations: strides (2, 4, 8, 8) to a 1024-dimensional latent (hop 512, about 86 frames per second at 44.1 kHz) and rates (8, 8, 4, 2) from a 1536-wide decoder to a tanh waveform. The quantizer (§3.2, App. A) projects each residual to 8 dimensions, picks the nearest of 1024 codes by cosine similarity, and projects back; its codebooks learn by the VQ-VAE codebook and commitment losses. Quantizer dropout (§3.3) draws n_q ~ U[1, 9] for each example.
Training (§3.4–3.5, §4.3) minimizes 15 · the L1 distance of log-mel spectrograms over seven windows (32 … 2048 samples, 5 … 320 mel bins) + 2 · feature matching + 1 · the hinge adversarial loss + 1 · codebook + 0.25 · commitment, against five period discriminators (2, 3, 5, 7, 11) and complex STFT discriminators at windows 2048, 1024 and 512 split into five frequency bands. AdamW (1e-4, β = 0.8, 0.9) trains both, its rate decaying by 0.999996 each step.
Constructors
| Constructor | Summary |
|---|---|
DAC(NeuralNetworkArchitecture<T> architecture, DACOptions options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer) |
Creates a trainable DAC. |
DAC(NeuralNetworkArchitecture<T> architecture, string modelPath, DACOptions options) |
Creates a trainable DAC. |
Properties
| Property | Summary |
|---|---|
int HopLength { get; } |
Methods
| Method | Summary |
|---|---|
ModelMetadata<T> GetModelMetadata() |
|
void LoadDescriptCheckpoint(string path) |
Loads descript-audio-codec’s .pth checkpoint (its state_dict entry, or a bare state dict) in descript’s layout; see LoadDescriptWeights. |
void LoadPretrainedWeights(string path) |
Loads the official DAC weights from a safetensors file in the Hugging Face layout (for example descript/dac_44khz’s model.safetensors) into this model’s encoder, quantizer and decoder. |