EnCodecOptions

Options & Configuration · AiDotNet.Audio.Generation

Options for EnCodec (“High Fidelity Neural Audio Compression”, Défossez et al. 2022). The defaults are the paper’s 24 kHz streamable model; OfficialCheckpoint24kHz and OfficialCheckpoint48kHz are the configurations of the released checkpoints, which differ from the paper’s text where noted.

Inheritance: ModelOptions → NeuralAudioCodecOptions → EnCodecOptions

For Beginners

EnCodec turns audio into a short sequence of numbers (codes) and back. The number of codebooks used per frame sets the bitrate: at 24 kHz each codebook adds 0.75 kbps, so 8 codebooks give 6 kbps.

Constructors

Constructor Summary
EnCodecOptions() Creates the paper’s 24 kHz streamable configuration.

Properties

Property Summary
bool Causal { get; set; } Gets or sets whether convolution padding is causal (true at 24 kHz).
bool CommitmentAsMean { get; set; } Gets or sets whether the commitment loss is the mean over codebooks of the mean squared error (the reference code) rather than Eq.
bool DiscriminatorPerBandwidth { get; set; } Gets or sets whether each bandwidth has its own discriminator (true: “a dedicated discriminator per-bandwidth”).
bool NormalizeVolume { get; set; } Gets or sets whether each chunk is divided by its RMS before encoding and multiplied back after decoding (false at 24 kHz; the 48 kHz model’s “normalize each chunk”).
bool ReflectPadding { get; set; } Gets or sets whether convolution padding reflects the signal rather than zero-filling (true; reference).
bool TrueSkip { get; set; } Gets or sets whether the residual shortcut is the identity rather than a 1×1 convolution (false; reference).
bool UseBalancer { get; set; } Gets or sets whether the balancer rescales the losses’ gradients (true); false backpropagates Σ λ_i ℓ_i.
bool? TimeGroupNormalization { get; set; } Overrides convolution normalization.
double AdversarialLossWeight { get; set; } Gets or sets λ_g, the adversarial weight (3 at 24 kHz, 4 at 48 kHz).
double BalancerDecay { get; set; } Gets or sets the balancer’s EMA decay β (0.999).
double BalancerTotalNorm { get; set; } Gets or sets the balancer’s reference norm R (1).
double Beta1 { get; set; } Gets or sets Adam’s β1 (0.5).
double Beta2 { get; set; } Gets or sets Adam’s β2 (0.9).
double ChunkOverlap { get; set; } Gets or sets the overlap between chunks as a fraction of a chunk (0.01: “an overlap of 10 ms”).
double CodebookDecay { get; set; } Gets or sets the codebooks’ EMA decay (0.99).
double CommitmentLossWeight { get; set; } Gets or sets λ_w, the commitment weight, applied outside the balancer (1).
double DiscriminatorSlope { get; set; } Gets or sets the discriminator’s LeakyReLU slope (0.2; reference).
double DiscriminatorUpdateProbability { get; set; } Gets or sets the probability a step updates the discriminator (2/3 at 24 kHz, 0.5 at 48 kHz).
double FeatureLossWeight { get; set; } Gets or sets λ_feat, the relative feature-matching weight (3 at 24 kHz, 4 at 48 kHz).
double FrequencyLossWeight { get; set; } Gets or sets λ_f, the multi-scale mel weight (1).
double LearningRate { get; set; } Gets or sets Adam’s learning rate (3e-4).
double MelL2Coefficient { get; set; } Gets or sets α_i, the L2 coefficient of every scale (1).
double MelMinFrequency { get; set; } Gets or sets the mel losses’ lowest frequency (64 Hz; the authors’ training configuration, as the paper does not state it).
double TimeLossWeight { get; set; } Gets or sets λ_t, the time-domain L1 weight (0.1).
double? ChunkSeconds { get; set; } Gets or sets the chunk length in seconds the encoder processes separately (null: the whole input; 1 s for the non-streamable 48 kHz model).
double[] TargetBandwidths { get; set; } Gets or sets the bandwidths (kbps) trained, one drawn per step (1.5, 3, 6, 12, 24 at 24 kHz).
int Compress { get; set; } Gets or sets the residual unit’s channel reduction (2; reference).
int DeadCodeThreshold { get; set; } Gets or sets the EMA usage below which a code is replaced by a batch vector (2; reference).
int DilationBase { get; set; } Gets or sets the dilation growth over a block’s residual units (2; reference).
int Dimension { get; set; } Gets or sets the latent dimension D (128).
int DiscriminatorFilters { get; set; } Gets or sets the discriminator’s channels (32).
int DiscriminatorKernelFrequency { get; set; } Gets or sets the kernel along frequency (9: the paper’s Fig.
int DiscriminatorKernelTime { get; set; } Gets or sets the kernel along time (3).
int Filters { get; set; } Gets or sets the base channel count C (32).
int KMeansIterations { get; set; } Gets or sets the k-means iterations of the first-batch codebook initialization (50; reference).
int KernelSize { get; set; } Gets or sets the first convolution’s kernel (7).
int LastKernelSize { get; set; } Gets or sets the last convolution’s kernel (7).
int LstmLayers { get; set; } Gets or sets the LSTM layers after the encoder’s and before the decoder’s convolutions (2).
int MelBins { get; set; } Gets or sets the mel bins (64).
int ResidualLayers { get; set; } Gets or sets the residual units per block (1).
int[] DiscriminatorDilations { get; set; } Gets or sets the time dilations of the strided convolutions (1, 2, 4).
int[] DiscriminatorWindows { get; set; } Gets or sets the MS-STFT discriminator’s STFT windows (2048, 1024, 512, 256, 128; doubled at 48 kHz).
int[] MelScales { get; set; } Gets or sets the mel losses’ scales i (window 2^i, hop 2^i / 4): 5 … 11.
int[] Ratios { get; set; } Gets or sets the strides in decoder order (8, 5, 4, 2); the encoder applies them reversed (2, 4, 5, 8), 320× at 24 kHz (75 frames per second).
int[] ResidualKernelSizes { get; set; } Gets or sets the residual unit’s two kernels: the paper’s “two convolutions with kernel size 3” (3, 3); the released checkpoints use (3, 1).

Methods

Method Summary
EnCodecOptions OfficialCheckpoint24kHz() The configuration of the released 24 kHz checkpoint (encodec_24khz): the paper’s defaults with the residual kernels (3, 1).
EnCodecOptions OfficialCheckpoint48kHz() The configuration of the released 48 kHz stereo checkpoint (encodec_48khz): non-streamable, layer-normalized, 1 s chunks with volume normalization, 16 codebooks (3 to 24 kbps), λ_g = λ_feat = 4, doubled discriminator windows updated with…