WaveGrad<T>
Models & Types · AiDotNet.TextToSpeech.Vocoders
WaveGrad: estimating gradients for waveform generation — a diffusion vocoder whose noise predictor is conditioned on the continuous noise level √ᾱ, built from GAN-TTS-style upsampling blocks over the mel spectrogram modulated by FiLM from downsampling blocks over the noisy waveform.
Inheritance: NeuralNetworkBase<T> → TtsModelBase<T> → VocoderBase<T> → SegmentVocoderBase<T> → DiffusionVocoderBase<T> → WaveGrad<T>
For Beginners
WaveGrad starts from noise and repeatedly nudges it toward speech, each step following the network’s estimate of which direction makes the audio more like real speech for this spectrogram.
How It Works
References: “WaveGrad: Estimating Gradients for Waveform Generation” (Chen et al., ICLR 2021) and lmnt-com/wavegrad for what the paper leaves unstated.
Training (Algorithm 1, §2.2): a segment s uniform over 1..S, a level √ᾱ uniform between l_{s−1} and l_s
(l_0 = 1, l_s = √Π_{i≤s}(1 − β_i)), y = √ᾱ y₀ + √(1 − ᾱ) ε and the L1 loss
‖ε − ε_θ(y, x, √ᾱ)‖₁. Synthesis (Algorithm 2) runs the reverse process over any β schedule, conditioning each
step on that schedule’s √ᾱ_n, so one model serves the 1000-step and the six-step schedules alike.
Example
var architecture = new NeuralNetworkArchitecture<double>(
inputType: InputType.OneDimensional,
taskType: NeuralNetworkTaskType.Regression,
inputSize: 128, outputSize: 300);
// The paper's WaveGrad Base, sampled with a short six-step schedule.
var model = new WaveGrad<double>(architecture,
new WaveGradOptions { InferenceNoiseSchedule = [1e-6, 1e-5, 1e-4, 1e-3, 1e-2, 1e-1] });
Constructors
| Constructor | Summary |
|---|---|
WaveGrad(NeuralNetworkArchitecture<T> architecture, WaveGradOptions options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer) |
Creates a trainable WaveGrad. |
WaveGrad(NeuralNetworkArchitecture<T> architecture, string modelPath, WaveGradOptions options) |
Creates a trainable WaveGrad. |
Properties
| Property | Summary |
|---|---|
int UpsampleFactor { get; } |
Methods
| Method | Summary |
|---|---|
ModelMetadata<T> GetModelMetadata() |