VarianceAdaptorTtsModelBase<T>
Base Classes · AiDotNet.TextToSpeech
Base class for non-autoregressive acoustic models of the FastSpeech family: a phoneme encoder, a variance adaptor (duration and length regulation, optionally pitch and energy), and a mel decoder.
Inheritance: NeuralNetworkBase<T> → TtsModelBase<T> → VarianceAdaptorTtsModelBase<T>
Known derived types: AdaSpeech<T>, FastSpeech2<T>, FastSpeech<T>, ProDiff<T>, PromptTTS<T>
How It Works
The layer stack is encoder layers, , decoder layers, registered with
AddEncoderDecoderLayers so the adaptor sits at Layers[EncoderLayerCount].
Inference is an ordinary sequential forward (the adaptor uses its predictions). Training is the papers’
teacher-forced objective: forced-alignment durations, and pitch and energy when the adaptor has those branches,
drive the adaptor, and the loss is the mel reconstruction error plus the mean squared error of each predictor
(FastSpeech, Ren et al. 2019 §3.3; FastSpeech 2, Ren et al. 2021 §2.3).
A model supplies its layers and its paper’s mel loss; the training and inference plumbing is shared.
Properties
| Property | Summary |
|---|---|
VarianceAdaptorLayer<T> VarianceAdaptor { get; } |
The variance adaptor between the phoneme encoder and the mel decoder. |
Methods
| Method | Summary |
|---|---|
T EvaluateTrainingObjective(TtsTrainingSample<T> sample) |
|
void Train(Tensor<T> input, Tensor<T> expectedOutput) |