ParallelWaveGAN<T>

Models & Types · AiDotNet.TextToSpeech.Vocoders

Parallel WaveGAN: a non-autoregressive WaveNet that turns Gaussian noise into speech conditioned on a mel spectrogram, trained jointly with a multi-resolution STFT loss and an adversarial loss.

Inheritance: NeuralNetworkBase<T> → TtsModelBase<T> → VocoderBase<T> → GanVocoderBase<T> → ParallelWaveGAN<T>

For Beginners

Parallel WaveGAN shapes random noise into speech in a single pass, guided by the spectrogram; matching spectrograms at several resolutions teaches it quickly, and a critic polishes the result.

How It Works

References: “Parallel WaveGAN: A fast waveform generation model based on generative adversarial networks with multi-resolution spectrogram” (Yamamoto et al., ICASSP 2020) and kan-bayashi/ParallelWaveGAN for what the paper leaves unstated.

The generator is trained on L_aux + λ_adv · mean((1 − D(G(z)))²) (Eq. 1, 7) with λ_adv = 4, where L_aux averages the spectral-convergence and log-magnitude losses over three STFT resolutions (Eq. 3–6); for the first 100k steps the discriminator is fixed and the generator trains on L_aux alone. The discriminator then trains on mean((1 − D(x))²) + mean(D(G(z))²) (Eq. 2) after each generator step, on a fresh generation. Both use RAdam (ε = 1e-6), at 1e-4 and 5e-5, halved every 200k steps.

Constructors

Constructor Summary
ParallelWaveGAN(NeuralNetworkArchitecture<T> architecture, ParallelWaveGANOptions options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer) Creates a trainable Parallel WaveGAN.
ParallelWaveGAN(NeuralNetworkArchitecture<T> architecture, string modelPath, ParallelWaveGANOptions options) Creates a trainable Parallel WaveGAN.

Properties

Property Summary
int UpsampleFactor { get; }

Methods

Method Summary
ModelMetadata<T> GetModelMetadata()