WaveGrad<T>

Models & Types · AiDotNet.TextToSpeech.Vocoders

WaveGrad: estimating gradients for waveform generation — a diffusion vocoder whose noise predictor is conditioned on the continuous noise level √ᾱ, built from GAN-TTS-style upsampling blocks over the mel spectrogram modulated by FiLM from downsampling blocks over the noisy waveform.

Inheritance: NeuralNetworkBase<T> → TtsModelBase<T> → VocoderBase<T> → SegmentVocoderBase<T> → DiffusionVocoderBase<T> → WaveGrad<T>

For Beginners

WaveGrad starts from noise and repeatedly nudges it toward speech, each step following the network’s estimate of which direction makes the audio more like real speech for this spectrogram.

How It Works

References: “WaveGrad: Estimating Gradients for Waveform Generation” (Chen et al., ICLR 2021) and lmnt-com/wavegrad for what the paper leaves unstated.

Training (Algorithm 1, §2.2): a segment s uniform over 1..S, a level √ᾱ uniform between l_{s−1} and l_s (l_0 = 1, l_s = √Π_{i≤s}(1 − β_i)), y = √ᾱ y₀ + √(1 − ᾱ) ε and the L1 loss ‖ε − ε_θ(y, x, √ᾱ)‖₁. Synthesis (Algorithm 2) runs the reverse process over any β schedule, conditioning each step on that schedule’s √ᾱ_n, so one model serves the 1000-step and the six-step schedules alike.

Example

var architecture = new NeuralNetworkArchitecture<double>(
    inputType: InputType.OneDimensional,
    taskType: NeuralNetworkTaskType.Regression,
    inputSize: 128, outputSize: 300);
            
// The paper's WaveGrad Base, sampled with a short six-step schedule.
var model = new WaveGrad<double>(architecture,
    new WaveGradOptions { InferenceNoiseSchedule = [1e-6, 1e-5, 1e-4, 1e-3, 1e-2, 1e-1] });

Constructors

Constructor Summary
WaveGrad(NeuralNetworkArchitecture<T> architecture, WaveGradOptions options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer) Creates a trainable WaveGrad.
WaveGrad(NeuralNetworkArchitecture<T> architecture, string modelPath, WaveGradOptions options) Creates a trainable WaveGrad.

Properties

Property Summary
int UpsampleFactor { get; }

Methods

Method Summary
ModelMetadata<T> GetModelMetadata()