MultiBandMelGAN<T>
Models & Types · AiDotNet.TextToSpeech.Vocoders
Multi-band MelGAN: a MelGAN generator that predicts PQMF sub-bands of the waveform at a quarter of its rate, trained with full- and sub-band multi-resolution STFT losses and multi-scale discriminators.
Inheritance: NeuralNetworkBase<T> → TtsModelBase<T> → VocoderBase<T> → GanVocoderBase<T> → MultiBandMelGAN<T>
For Beginners
Instead of generating every audio sample, the network generates four narrow frequency bands at a quarter of the sample rate and a fixed filter bank merges them, which makes it several times faster.
How It Works
References: “Multi-band MelGAN: Faster Waveform Generation for High-Quality Text-to-Speech” (Yang et al., SLT 2021) and kan-bayashi/ParallelWaveGAN for what the paper leaves unstated.
The generator predicts four sub-band signals; the PQMF synthesis filter merges them into the full-band waveform the
discriminators see (§2.2, §3.2). The generator first trains alone for 200k steps on
L_mr_stft = ½ (L_full + L_sub) (Eq. 9; sub-band targets from the PQMF analysis filter), then the
discriminators train on the LSGAN loss (Eq. 1) and the generator on λ_adv Σ_k (D_k(G(s)) − 1)² + L_mr_stft
(Eq. 8). Both use Adam at 1e-4, halved every 100k steps to 1e-6.
Constructors
| Constructor | Summary |
|---|---|
MultiBandMelGAN(NeuralNetworkArchitecture<T> architecture, MultiBandMelGANOptions options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer) |
Creates a trainable Multi-band MelGAN. |
MultiBandMelGAN(NeuralNetworkArchitecture<T> architecture, string modelPath, MultiBandMelGANOptions options) |
Creates a trainable Multi-band MelGAN. |
Properties
| Property | Summary |
|---|---|
int UpsampleFactor { get; } |
Methods
| Method | Summary |
|---|---|
ModelMetadata<T> GetModelMetadata() |