HiFiGAN<T>
Models & Types · AiDotNet.TextToSpeech.Vocoders
HiFi-GAN: a GAN vocoder whose generator upsamples a mel spectrogram to a waveform through transposed convolutions and multi-receptive-field residual blocks, trained against multi-period and multi-scale discriminators.
Inheritance: NeuralNetworkBase<T> → TtsModelBase<T> → VocoderBase<T> → GanVocoderBase<T> → HiFiGAN<T>
For Beginners
A generator turns a spectrogram into audio while two “critics” — one looking at periodic patterns, one at different time scales — learn to tell its audio from real recordings; competing makes the audio realistic.
How It Works
References: “HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesis” (Kong et al., NeurIPS 2020) and its reference implementation (jik876/hifi-gan) for what the paper leaves unstated.
Each training step (§2.4, reference train.py) crops an 8192-sample segment, computes its log-mel input, runs
the generator, updates the discriminators on the LSGAN loss Σ_k (D_k(x) − 1)² + D_k(G(s))² (Eq. 1, 6) with
the generation detached, then updates the generator on Σ_k (D_k(G(s)) − 1)² + 2 L_FM + 45 L_Mel (Eq. 2–5, 7),
where L_FM is the mean L1 distance of every discriminator feature map and L_Mel the L1 distance of log-mel
spectrograms (full band). Generator and discriminators each use AdamW (β = 0.8, 0.99, weight decay 0.01) at 2e-4,
decayed by 0.999 every epoch. The input mel is the reference’s mel_spectrogram of the audio
(ComputeMel); synthesis expects the same features.
Constructors
| Constructor | Summary |
|---|---|
HiFiGAN(NeuralNetworkArchitecture<T> architecture, HiFiGANOptions options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer) |
Creates a trainable HiFi-GAN. |
HiFiGAN(NeuralNetworkArchitecture<T> architecture, string modelPath, HiFiGANOptions options) |
Creates a trainable HiFi-GAN. |
Properties
| Property | Summary |
|---|---|
int UpsampleFactor { get; } |
Methods
| Method | Summary |
|---|---|
ModelMetadata<T> GetModelMetadata() |