HiFiGAN<T>

Models & Types · AiDotNet.TextToSpeech.Vocoders

HiFi-GAN: a GAN vocoder whose generator upsamples a mel spectrogram to a waveform through transposed convolutions and multi-receptive-field residual blocks, trained against multi-period and multi-scale discriminators.

Inheritance: NeuralNetworkBase<T> → TtsModelBase<T> → VocoderBase<T> → GanVocoderBase<T> → HiFiGAN<T>

For Beginners

A generator turns a spectrogram into audio while two “critics” — one looking at periodic patterns, one at different time scales — learn to tell its audio from real recordings; competing makes the audio realistic.

How It Works

References: “HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesis” (Kong et al., NeurIPS 2020) and its reference implementation (jik876/hifi-gan) for what the paper leaves unstated.

Each training step (§2.4, reference train.py) crops an 8192-sample segment, computes its log-mel input, runs the generator, updates the discriminators on the LSGAN loss Σ_k (D_k(x) − 1)² + D_k(G(s))² (Eq. 1, 6) with the generation detached, then updates the generator on Σ_k (D_k(G(s)) − 1)² + 2 L_FM + 45 L_Mel (Eq. 2–5, 7), where L_FM is the mean L1 distance of every discriminator feature map and L_Mel the L1 distance of log-mel spectrograms (full band). Generator and discriminators each use AdamW (β = 0.8, 0.99, weight decay 0.01) at 2e-4, decayed by 0.999 every epoch. The input mel is the reference’s mel_spectrogram of the audio (ComputeMel); synthesis expects the same features.

Constructors

Constructor Summary
HiFiGAN(NeuralNetworkArchitecture<T> architecture, HiFiGANOptions options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer) Creates a trainable HiFi-GAN.
HiFiGAN(NeuralNetworkArchitecture<T> architecture, string modelPath, HiFiGANOptions options) Creates a trainable HiFi-GAN.

Properties

Property Summary
int UpsampleFactor { get; }

Methods

Method Summary
ModelMetadata<T> GetModelMetadata()