Vocos<T>
Models & Types · AiDotNet.TextToSpeech.Vocoders
Vocos: a GAN vocoder that generates Fourier spectral coefficients — magnitude and phase — at frame rate with a ConvNeXt backbone and turns them into audio with the inverse STFT.
Inheritance: NeuralNetworkBase<T> → TtsModelBase<T> → VocoderBase<T> → GanVocoderBase<T> → Vocos<T>
For Beginners
Rather than building audio sample by sample, Vocos predicts how loud and in what phase each frequency is in every frame and lets an inverse Fourier transform assemble the sound, which is very fast.
How It Works
References: “Vocos: Closing the gap between time-domain and Fourier-based neural vocoders for high-quality audio synthesis” (Siuzdak, ICLR 2024) and gemelo-ai/vocos for what the paper leaves unstated.
The head’s magnitude is exp(m) and its phase the point (cos p, sin p) on the unit circle, wrapping
implicitly (§3.2). The discriminators train on the hinge loss 1/K Σ_k max(0, 1 − D_k(x)) + max(0, 1 + D_k(x̂)) and
the generator on 1/K Σ_k max(0, 1 − D_k(x̂)) plus feature matching and the mel L1 (§3.3), the multi-resolution
discriminator’s terms weighted 0.1 and the mel term 45 as the reference trains it. AdamW (0.9, 0.999) at 2e-4 with a
cosine decay (§4.1).
Constructors
| Constructor | Summary |
|---|---|
Vocos(NeuralNetworkArchitecture<T> architecture, VocosOptions options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer) |
Creates a trainable Vocos. |
Vocos(NeuralNetworkArchitecture<T> architecture, string modelPath, VocosOptions options) |
Creates a trainable Vocos. |
Properties
| Property | Summary |
|---|---|
int UpsampleFactor { get; } |
Methods
| Method | Summary |
|---|---|
ModelMetadata<T> GetModelMetadata() |