SegmentVocoderBase<T>
Base Classes · AiDotNet.TextToSpeech.Vocoders
The training and synthesis plumbing of vocoders trained by one objective on aligned (mel spectrogram, waveform) segments — the diffusion vocoders (DiffWave, WaveGrad, PriorGrad, FreGrad), the flow WaveGlow and the autoregressive WaveNet and WaveRNN: a model supplies its network, input features, segment length, objective and synthesis.
Inheritance: NeuralNetworkBase<T> → TtsModelBase<T> → VocoderBase<T> → SegmentVocoderBase<T>
Known derived types: DiffusionVocoderBase<T>, WaveGlow<T>, WaveNet<T>, WaveRNN<T>
For Beginners
This is the shared machinery: cut a matching piece of spectrogram and audio, compute the model’s loss on it, update the weights; to make speech, run the model’s own generation procedure.
How It Works
Training takes a frame-aligned segment of SegmentSize samples (rounded down to whole frames)
from a random frame, and one optimizer step on TrainingObjective. The random draws of a step (noise
levels, noise, dropout of the objective) come from one seed per step, so the objective is a deterministic function
of the parameters within the step. Synthesis runs with the model’s sampling seed, so it is repeatable.
Methods
| Method | Summary |
|---|---|
IReadOnlyList<OutputAxisContract> OutputAxesFor(int inputRank) |
|
Tensor<T> ComputeMel(Tensor<T> audio) |
The network’s input features for audio``[samples], [1, mel, frames]. |
Tensor<T> MelToWaveform(Tensor<T> melSpectrogram) |
|
void Train(Tensor<T> input, Tensor<T> expectedOutput) |