UnivNet<T>
Models & Types · AiDotNet.TextToSpeech.Vocoders
UnivNet: a GAN vocoder whose generator uses location-variable convolutions with kernels predicted from the mel spectrogram, trained against multi-resolution spectrogram and multi-period waveform discriminators.
Inheritance: NeuralNetworkBase<T> → TtsModelBase<T> → VocoderBase<T> → GanVocoderBase<T> → UnivNet<T>
For Beginners
Every short stretch of the spectrogram designs its own small filter, which the network uses to shape noise into the matching stretch of audio; critics looking at spectrograms of several resolutions keep the result sharp.
How It Works
References: “UnivNet: A Neural Vocoder with Multi-Resolution Spectrogram Discriminators for High-Fidelity Waveform Generation” (Jang et al., Interspeech 2021) and maum-ai/univnet for what the paper leaves unstated.
The generator shapes Gaussian noise into a waveform through three LVC residual stacks (§3.1). It first trains alone for 200k steps on λ · L_aux, the multi-resolution STFT loss (§3.3, §4.3); afterwards each step updates the generator on λ · L_aux plus the LSGAN generator loss averaged over the three resolution and five period discriminators (§3.2), and the discriminators on the averaged LSGAN loss against that same generation. Adam (β = 0.5, 0.9) at 1e-4.
Constructors
| Constructor | Summary |
|---|---|
UnivNet(NeuralNetworkArchitecture<T> architecture, UnivNetOptions options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer) |
Creates a trainable UnivNet. |
UnivNet(NeuralNetworkArchitecture<T> architecture, string modelPath, UnivNetOptions options) |
Creates a trainable UnivNet. |
Properties
| Property | Summary |
|---|---|
int UpsampleFactor { get; } |
Methods
| Method | Summary |
|---|---|
ModelMetadata<T> GetModelMetadata() |