DiffusionVocoderBase<T>
Base Classes · AiDotNet.TextToSpeech.Vocoders
The denoising-diffusion training and sampling diffusion vocoders share (Ho et al. 2020; DiffWave, WaveGrad, PriorGrad, FreGrad): a network ε_θ predicts the noise added to a waveform at a noise level, conditioned on the mel spectrogram, and the reverse process turns noise into speech.
Inheritance: NeuralNetworkBase<T> → TtsModelBase<T> → VocoderBase<T> → SegmentVocoderBase<T> → DiffusionVocoderBase<T>
Known derived types: DiffWave<T>, FreGrad<T>, PriorGrad<T>, WaveGrad<T>
For Beginners
The model learns to remove a little noise at a time; to make speech it starts from pure noise and removes noise step by step, guided by the spectrogram.
How It Works
Training (Algorithm 1 of DDPM and the vocoder papers): crop a segment, draw a noise level, form
x_t = √ᾱ_t x₀ + √(1 − ᾱ_t) ε and minimize the model’s loss between ε and ε_θ(x_t, level, mel). Sampling
(Algorithm 2): from x_N ~ the prior, x_{n−1} = (x_n − β_n / √(1 − ᾱ_n) · ε_θ) / √α_n + σ_n z with
σ_n = √((1 − ᾱ_{n−1}) / (1 − ᾱ_n) · β_n), over the training schedule or a shorter inference schedule.
A model supplies its network (Denoise), its input features, its schedules and how the noise level
conditions the network, and may change the noise prior and the loss (PriorGrad’s data-dependent prior and Mahalanobis
loss) and the space the diffusion runs in (FreGrad’s wavelet sub-bands).