GradTTS<T>
Models & Types · AiDotNet.TextToSpeech.Classic
Grad-TTS: text-to-speech with a score-based diffusion decoder that turns an aligned Gaussian prior into a mel spectrogram.
Inheritance: NeuralNetworkBase<T> → TtsModelBase<T> → GradTTS<T>
Implements: IAcousticModel<T>
For Beginners
Grad-TTS first sketches a blurry spectrogram from the text, then refines it step by step by removing noise, the way image diffusion models paint pictures.
How It Works
References: “Grad-TTS: A Diffusion Probabilistic Model for Text-to-Speech” (Popov et al., ICML 2021) and its reference implementation (huawei-noah/Speech-Backbones) for what the paper leaves unstated.
The encoder is Glow-TTS’s (§3): it predicts per-token prior means μ and durations, aligned to the mel spectrogram
by monotonic alignment search during training. The decoder is a score network s_θ(X_t, μ, t)
(GradTtsScoreEstimator, a U-Net) for the forward diffusion
dX = ½ β_t (μ − X) dt + √β_t dW with β_t = 0.05 + 19.95 t, whose terminal distribution is N(μ, I).
Training minimizes the duration loss, the prior loss ½‖y − μ‖² and the score-matching loss on random 2-second
segments (MAS and durations use the whole utterance), with encoder and decoder gradients each clipped to norm 1.
Inference draws X₁ = μ + ε/τ (τ = 1.5) and solves the reverse ODE with N Euler steps.
The model aligns itself, so a plain Train(tokens, mel) is its full training signal.
Constructors
| Constructor | Summary |
|---|---|
GradTTS(NeuralNetworkArchitecture<T> architecture, GradTTSOptions options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer) |
|
GradTTS(NeuralNetworkArchitecture<T> architecture, string modelPath, GradTTSOptions options) |
Properties
| Property | Summary |
|---|---|
int FftSize { get; } |
|
int HopSize { get; } |
|
int MaxTextLength { get; } |
|
int MelChannels { get; } |
Methods
| Method | Summary |
|---|---|
ModelMetadata<T> GetModelMetadata() |
|
T EvaluateTrainingObjective(TtsTrainingSample<T> sample) |
|
Tensor<T> TextToMel(string text) |
Generates a mel spectrogram from text (the paper pairs it with HiFi-GAN). |