ForwardTacotron<T>
Models & Types · AiDotNet.TextToSpeech.Classic
ForwardTacotron: a non-autoregressive Tacotron that predicts phoneme durations, pitch and energy and generates the whole mel spectrogram in one pass.
Inheritance: NeuralNetworkBase<T> → TtsModelBase<T> → ForwardTacotron<T>
Implements: IAcousticModel<T>
For Beginners
ForwardTacotron decides how long, how high and how loud each sound is, then paints the whole spectrogram at once, so it never skips or repeats words.
How It Works
Reference: ForwardTacotron (Axel Springer AI, as-ideas/ForwardTacotron), a model published as a reference implementation rather than a paper; it is inspired by FastSpeech and built from Tacotron components. The autoregressive Non-Attentive Tacotron (Shen et al. 2020) is a different model.
Three series predictors (phoneme embedding, three kernel-5 convolutions with ReLU and batch normalization, a
bidirectional GRU, a linear projection) predict each phoneme’s duration in frames and its normalized pitch and energy.
The mel path embeds the phonemes, runs a CBHG pre-net (ForwardTacotron), adds the pitch and
energy series through kernel-3 convolutions, expands by the durations, and decodes with a bidirectional LSTM, a linear
projection and a CBHG post-net with a bias-free projection. Training uses the ground-truth durations (from a
Tacotron’s attention), phoneme-level pitch (mean voiced WORLD F0 in 30–600 Hz) and energy (mean L2 norm of exp(mel)),
with L1 losses on mel, post-net mel and 0.1 × each series.
Durations come from a forced alignment outside the model, so a plain Train(tokens, mel) throws; pass a
TtsTrainingSample with Durations.
Constructors
| Constructor | Summary |
|---|---|
ForwardTacotron(NeuralNetworkArchitecture<T> architecture, ForwardTacotronOptions options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer) |
|
ForwardTacotron(NeuralNetworkArchitecture<T> architecture, string modelPath, ForwardTacotronOptions options) |
Properties
| Property | Summary |
|---|---|
int FftSize { get; } |
|
int HopSize { get; } |
|
int MaxTextLength { get; } |
|
int MelChannels { get; } |
Methods
| Method | Summary |
|---|---|
ModelMetadata<T> GetModelMetadata() |
|
T EvaluateTrainingObjective(TtsTrainingSample<T> sample) |
|
Tensor<T> TextToMel(string text) |
Generates a log-mel spectrogram from text (pair it with a vocoder). |