ISTFTNet<T>
Models & Types · AiDotNet.TextToSpeech.Vocoders
iSTFTNet: a fast, lightweight mel-spectrogram vocoder — HiFi-GAN with its output-side upsampling replaced by an inverse STFT of a small magnitude and phase spectrogram it predicts.
Inheritance: NeuralNetworkBase<T> → TtsModelBase<T> → VocoderBase<T> → GanVocoderBase<T> → ISTFTNet<T>
For Beginners
Instead of building every audio sample with neural layers, the network predicts a tiny spectrogram and a fixed mathematical transform turns it into sound, which is much faster.
How It Works
References: “iSTFTNet: Fast and Lightweight Mel-Spectrogram Vocoder Incorporating Inverse Short-Time Fourier Transform” (Kaneko et al., ICASSP 2022).
After two ×8 upsamplings the output convolution gives (f/2 + 1) × 2 channels; an exponential makes the first half a linear magnitude and a sine makes the second half the phase (§3.3), and iSTFT(16, 4, 16) turns them into the waveform (Eq. 1). Training is HiFi-GAN’s — its discriminators, LSGAN, feature matching (×2) and mel L1 (×45) — with Adam (β = 0.5, 0.9) at 2e-4 (§4.1).
Constructors
| Constructor | Summary |
|---|---|
ISTFTNet(NeuralNetworkArchitecture<T> architecture, ISTFTNetOptions options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer) |
Creates a trainable iSTFTNet. |
ISTFTNet(NeuralNetworkArchitecture<T> architecture, string modelPath, ISTFTNetOptions options) |
Creates a trainable iSTFTNet. |
Properties
| Property | Summary |
|---|---|
int UpsampleFactor { get; } |
Methods
| Method | Summary |
|---|---|
ModelMetadata<T> GetModelMetadata() |