SpeedySpeech<T>
Models & Types · AiDotNet.TextToSpeech.Classic
SpeedySpeech: a fully convolutional non-autoregressive text-to-speech student network trained on durations from a teacher’s attention.
Inheritance: NeuralNetworkBase<T> → TtsModelBase<T> → SpeedySpeech<T>
Implements: IAcousticModel<T>
For Beginners
SpeedySpeech decides how long each phoneme lasts, stretches the phonemes to that many frames, and paints the whole spectrogram at once with stacks of convolutions — no attention, so it is fast.
How It Works
References: “SpeedySpeech: Efficient Neural Speech Synthesis” (Vainer & Dušek, Interspeech 2020), and the authors’ implementation (github.com/janvainer/speedyspeech) for what the paper leaves unstated.
The student (§3.2, Fig. 3) has three convolutional modules built from DilatedResidualConvBlock:
a phoneme encoder (embedding, a fully connected layer with ReLU, 13 two-convolution residual blocks with dilations
1, 2, 4 repeated, a long residual connection, and a normalized output projection), a duration predictor reading the
encodings with their gradient detached (“we detach gradient flow from the duration predictor to the encoder”), and
a decoder (17 blocks with dilations 1, 2, 4, 8 repeated) that turns the expanded encodings, plus a positional
encoding restarted at every phoneme, into a log-mel spectrogram. Training uses teacher durations for the expansion
and minimizes MAE + (1 − SSIM) on the standardized log-mel spectrogram plus the Huber loss of the log durations.
Training data: durations come from the teacher network (§3.1), so a plain Train(tokens, mel) throws; pass a
TtsTrainingSample with Durations.
Constructors
| Constructor | Summary |
|---|---|
SpeedySpeech(NeuralNetworkArchitecture<T> architecture, SpeedySpeechOptions options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer) |
|
SpeedySpeech(NeuralNetworkArchitecture<T> architecture, string modelPath, SpeedySpeechOptions options) |
Properties
| Property | Summary |
|---|---|
int FftSize { get; } |
|
int HopSize { get; } |
|
int MaxTextLength { get; } |
|
int MelChannels { get; } |
Methods
| Method | Summary |
|---|---|
ModelMetadata<T> GetModelMetadata() |
|
T EvaluateTrainingObjective(TtsTrainingSample<T> sample) |
|
Tensor<T> TextToMel(string text) |
Generates a log-mel spectrogram from text (pair it with a vocoder; the paper uses MelGAN). |