GanVocoderBase<T>

Base Classes · AiDotNet.TextToSpeech.Vocoders

The training loop GAN vocoders share: a generator that turns a mel spectrogram into a waveform, discriminators that tell real from generated audio, and alternating discriminator and generator steps on random segments.

Inheritance: NeuralNetworkBase<T> → TtsModelBase<T> → VocoderBase<T> → GanVocoderBase<T>

Known derived types: AmplitudePhaseVocoderBase<T>, BigVGAN<T>, HiFiGAN<T>, ISTFTNet<T>, MelGAN<T>, MultiBandMelGAN<T>, ParallelWaveGAN<T>, UnivNet<T>, Vocos<T>

For Beginners

A generator turns a spectrogram into audio while one or more “critics” learn to tell its audio from real recordings; competing makes the audio realistic.

How It Works

Each step crops a segment of SegmentSize samples (with its frames when the mel is supplied), updates the discriminators on a detached generation, and updates the generator — in the order the model’s reference uses (DiscriminatorStepFirst). A model can train its generator alone for its first steps (DiscriminatorStartStep, as Parallel WaveGAN and Multi-band MelGAN do). Each group has its own optimizer from CreatePaperOptimizer unless one was supplied at construction.

A model supplies its generator and discriminators, its input features (ComputeInputMel), its losses and its optimizer; the base owns segmenting, the step order, the parameter groups and the training objective the test harness and callers read.

Methods

Method Summary
IReadOnlyList<OutputAxisContract> OutputAxesFor(int inputRank)
Tensor<T> ComputeMel(Tensor<T> audio) The generator’s input features for audio``[samples], [1, mel, frames].
Tensor<T> MelToWaveform(Tensor<T> melSpectrogram)
void Train(Tensor<T> input, Tensor<T> expectedOutput)