SoundStreamOptions
Options & Configuration · AiDotNet.Audio.Generation
Options for SoundStream (“SoundStream: An End-to-End Neural Audio Codec”, Zeghidour et al. 2021). The defaults are the paper’s bitrate-scalable 24 kHz model: C = 32, strides (2, 4, 5, 8), 24 quantizers of 1024 codes (18 kbps) trained with quantizer dropout and used at 6 kbps.
Inheritance: ModelOptions → NeuralAudioCodecOptions → SoundStreamOptions
For Beginners
SoundStream compresses audio to a few numbers per 13 ms and rebuilds it. A single trained model works at any bitrate up to 18 kbps by using more or fewer codebooks per frame, and can optionally remove background noise while it compresses.
Constructors
| Constructor | Summary |
|---|---|
SoundStreamOptions() |
Creates the paper’s bitrate-scalable configuration. |
Properties
| Property | Summary |
|---|---|
SoundStreamFilmPosition FilmPosition { get; set; } |
Gets or sets where the FiLM denoising conditioning sits (Encoder). |
bool Denoise { get; set; } |
Gets or sets whether inference asks for denoising (false). |
bool QuantizerDropout { get; set; } |
Gets or sets whether each training example uses n_q ~ U[1, N_q] codebooks (quantizer dropout; true). |
bool ReflectPadding { get; set; } |
Gets or sets whether convolutions zero-pad the past rather than reflecting the signal (false: zeros, as causal padding of the past). |
double AdversarialLossWeight { get; set; } |
Gets or sets λ_adv (1). |
double Beta1 { get; set; } |
Gets or sets Adam’s β1 (0.5). |
double Beta2 { get; set; } |
Gets or sets Adam’s β2 (0.9). |
double CodebookDecay { get; set; } |
Gets or sets the codebooks’ EMA decay (0.99). |
double CommitmentLossWeight { get; set; } |
Gets or sets the commitment loss weight (0: Eq. |
double FeatureLossWeight { get; set; } |
Gets or sets λ_feat (100). |
double LearningRate { get; set; } |
Gets or sets Adam’s learning rate (3e-4; the paper states none, EnCodec’s reimplementation’s). |
double ReconstructionLossWeight { get; set; } |
Gets or sets λ_rec (1). |
int DeadCodeThreshold { get; set; } |
Gets or sets the EMA assignment count below which a code is replaced by a frame of the batch (2). |
int DilationBase { get; set; } |
Gets or sets the dilation growth over a block’s residual units (3: 1, 3, 9). |
int Dimension { get; set; } |
Gets or sets the embedding dimension D (the paper leaves its value unstated; 128, as EnCodec’s reimplementation). |
int Filters { get; set; } |
Gets or sets the base channel count C (C_enc = C_dec = 32). |
int KMeansIterations { get; set; } |
Gets or sets the k-means iterations of the first-batch initialization (50). |
int MelBins { get; set; } |
Gets or sets the mel bins (64). |
int ResidualLayers { get; set; } |
Gets or sets the residual units per block (3: dilations 1, 3, 9). |
int StftChannels { get; set; } |
Gets or sets the STFT discriminator’s base channels C (32). |
int StftHop { get; set; } |
Gets or sets the STFT discriminator’s hop H (256). |
int StftWindow { get; set; } |
Gets or sets the STFT discriminator’s window W (1024). |
int WaveDiscriminatorScales { get; set; } |
Gets or sets the wave-based discriminator’s resolutions (3: original, 2× and 4× down-sampled). |
int WaveDiscriminatorWidthDivisor { get; set; } |
Gets or sets a divisor on the wave-based discriminators’ widths (1: the paper’s). |
int[] MelScales { get; set; } |
Gets or sets the reconstruction loss’s scales i (window and FFT 2^i, hop 2^i / 4): 6 … 11. |
int[] Ratios { get; set; } |
Gets or sets the strides in decoder order (8, 5, 4, 2); the encoder applies them reversed (2, 4, 5, 8). |
int[] ResidualKernelSizes { get; set; } |
Gets or sets the residual unit’s kernels (7, then 1). |
int[] WaveDiscriminatorChannels { get; set; } |
Gets or sets the wave-based discriminators’ grouped-convolution widths (64, 256, 1024, 1024). |