GlowTTS<T>
Models & Types · AiDotNet.TextToSpeech.Classic
Glow-TTS: a flow-based generative text-to-speech model that finds its own alignment with monotonic alignment search.
Inheritance: NeuralNetworkBase<T> → TtsModelBase<T> → GlowTTS<T>
Implements: IAcousticModel<T>
For Beginners
Glow-TTS learns an exactly reversible mapping between spectrograms and simple Gaussian noise shaped by the text; to speak, it draws noise shaped by the text and runs the mapping backwards.
How It Works
References: “Glow-TTS: A Generative Flow for Text-to-Speech via Monotonic Alignment Search” (Kim et al., NeurIPS 2020) and its reference implementation (jaywalnut310/glow-tts) for what the paper leaves unstated.
The text encoder (§3.3, Fig. 7) embeds phonemes, refines them with a residual convolutional pre-net and a Transformer
encoder with relative position representations (RelativePositionTransformerBlock), and projects
each token to the mean of a Gaussian prior over mel frames (σ fixed at 1). The decoder (Fig. 8) is a normalizing
flow over pairs of frames squeezed into channels: blocks of ActNormFlowLayer,
GroupedInvertibleConvFlowLayer and AffineCouplingFlowLayer. Training maximizes
the exact log-likelihood of the mel spectrogram under the prior aligned by monotonic alignment search (§3.2), plus a
duration predictor’s log-domain MSE to the searched durations (on encoder outputs with their gradient stopped).
Inference samples the expanded prior at temperature 0.333 and runs the flow in reverse.
The model aligns itself, so a plain Train(tokens, mel) is its full training signal.
Constructors
| Constructor | Summary |
|---|---|
GlowTTS(NeuralNetworkArchitecture<T> architecture, GlowTTSOptions options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer) |
|
GlowTTS(NeuralNetworkArchitecture<T> architecture, string modelPath, GlowTTSOptions options) |
Properties
| Property | Summary |
|---|---|
int FftSize { get; } |
|
int HopSize { get; } |
|
int MaxTextLength { get; } |
|
int MelChannels { get; } |
Methods
| Method | Summary |
|---|---|
ModelMetadata<T> GetModelMetadata() |
|
T EvaluateTrainingObjective(TtsTrainingSample<T> sample) |
|
Tensor<T> TextToMel(string text) |
Generates a mel spectrogram from text (the paper pairs it with WaveGlow). |