NonAttentiveTacotron<T>
Models & Types · AiDotNet.TextToSpeech.Classic
Non-Attentive Tacotron: Tacotron 2 with its attention replaced by an explicit duration predictor and Gaussian upsampling (supervised-duration variant).
Inheritance: NeuralNetworkBase<T> → TtsModelBase<T> → NonAttentiveTacotron<T>
Implements: IAcousticModel<T>
For Beginners
Like Tacotron 2, but instead of letting attention guess where it is in the text, it is told how long each sound lasts, so it cannot skip or repeat words.
How It Works
References: “Non-Attentive Tacotron: Robust and Controllable Neural TTS Synthesis Including Unsupervised Duration Modeling” (Shen et al., 2020), §3 and Appendix A (Table 6).
The encoder embeds phonemes and applies three (dropout, batch normalization, convolution) layers and a bidirectional LSTM with zoneout. A duration predictor (two bidirectional LSTMs and a projection) predicts each token’s duration in seconds; a range predictor (two bidirectional LSTMs on the encoder output and the durations, a projection and SoftPlus) predicts σ. Gaussian upsampling (§3.1) spreads each encoder output over the frames with a Gaussian centred on its segment, and a sinusoidal embedding of each frame’s index within its token is concatenated. The autoregressive decoder feeds the previous frame through a two-layer pre-net, concatenates the current upsampled encoder output, runs two zoneout LSTMs, concatenates the upsampled output again and projects to r frames; a 5-layer post-net adds a residual. Training uses target durations for upsampling and minimizes L1 + L2 on the spectrogram before and after the post-net plus 2.0 × the squared duration error in seconds.
Durations come from outside the model (supervised training), so a plain Train(tokens, mel) throws; pass a
TtsTrainingSample with Durations (in frames).
Constructors
| Constructor | Summary |
|---|---|
NonAttentiveTacotron(NeuralNetworkArchitecture<T> architecture) |
Calls the fully specified overload with the omitted parameters’ defaults. |
NonAttentiveTacotron(NeuralNetworkArchitecture<T> architecture, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer) |
Calls the fully specified overload with the omitted parameters’ defaults. |
NonAttentiveTacotron(NeuralNetworkArchitecture<T> architecture, NonAttentiveTacotronOptions options) |
Calls the fully specified overload with the omitted parameters’ defaults. |
NonAttentiveTacotron(NeuralNetworkArchitecture<T> architecture, NonAttentiveTacotronOptions options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer) |
|
NonAttentiveTacotron(NeuralNetworkArchitecture<T> architecture, string modelPath) |
Calls the fully specified overload with the omitted parameters’ defaults. |
NonAttentiveTacotron(NeuralNetworkArchitecture<T> architecture, string modelPath, NonAttentiveTacotronOptions options) |
Properties
| Property | Summary |
|---|---|
int FftSize { get; } |
|
int HopSize { get; } |
|
int MaxTextLength { get; } |
|
int MelChannels { get; } |
Methods
| Method | Summary |
|---|---|
ModelMetadata<T> GetModelMetadata() |
|
T EvaluateTrainingObjective(TtsTrainingSample<T> sample) |
|
Tensor<T> TextToMel(string text) |
Generates a mel spectrogram from text (pair it with a vocoder). |