MelGAN<T>

Models & Types · AiDotNet.TextToSpeech.Vocoders

MelGAN: a non-autoregressive, fully convolutional GAN vocoder — transposed-convolution upsampling with dilated residual stacks — trained against multi-scale window-based discriminators with the hinge loss and feature matching.

Inheritance: NeuralNetworkBase<T> → TtsModelBase<T> → VocoderBase<T> → GanVocoderBase<T> → MelGAN<T>

For Beginners

A fast generator turns a spectrogram into audio while three critics, listening at different time scales, learn to tell its audio from real speech.

How It Works

References: “MelGAN: Generative Adversarial Networks for Conditional Waveform Synthesis” (Kumar et al., NeurIPS 2019) and its released code (descriptinc/melgan-neurips) for what the paper leaves unstated.

Each step crops an 8192-sample segment, updates the three discriminators on Σ_k mean(max(0, 1 − D_k(x))) + mean(max(0, 1 + D_k(G(s)))) (Eq. 1) with the generation detached, then the generator on Σ_k −mean(D_k(G(s))) + λ Σ_k Σ_i mean|D_k⁽ⁱ⁾(x) − D_k⁽ⁱ⁾(G(s))| with λ = 10 (Eq. 2–4) over every intermediate discriminator layer; no loss is taken in the audio or spectrogram domain (§2.3). Both networks use Adam (β = 0.5, 0.9) at 1e-4.

Constructors

Constructor Summary
MelGAN(NeuralNetworkArchitecture<T> architecture, MelGANOptions options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer) Creates a trainable MelGAN.
MelGAN(NeuralNetworkArchitecture<T> architecture, string modelPath, MelGANOptions options) Creates a trainable MelGAN.

Properties

Property Summary
int UpsampleFactor { get; }

Methods

Method Summary
ModelMetadata<T> GetModelMetadata()