WaveGlow<T>
Models & Types · AiDotNet.TextToSpeech.Vocoders
WaveGlow: a flow-based generative network for speech synthesis — Glow’s invertible 1×1 convolutions and affine couplings over groups of audio samples, with WaveNet-like coupling networks conditioned on the mel spectrogram, trained only by maximizing the likelihood.
Inheritance: NeuralNetworkBase<T> → TtsModelBase<T> → VocoderBase<T> → SegmentVocoderBase<T> → WaveGlow<T>
For Beginners
WaveGlow learns a reversible mapping between speech and random noise; to make speech it draws noise and runs the mapping backwards, guided by the spectrogram.
How It Works
References: “WaveGlow: A Flow-based Generative Network for Speech Synthesis” (Prenger et al., ICASSP 2019) and NVIDIA/waveglow for what the paper leaves unstated.
Training maximizes log p(x) = −z(x)ᵀz(x) / 2σ² + Σ log s + Σ log|det W| (Eq. 12) of a clip, squeezed into groups
of 8 samples and passed through the steps of flow with early outputs (§2.3). Synthesis draws z with a smaller
σ (§2.4) and inverts every step.
Constructors
| Constructor | Summary |
|---|---|
WaveGlow(NeuralNetworkArchitecture<T> architecture, WaveGlowOptions options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer) |
Creates a trainable WaveGlow. |
WaveGlow(NeuralNetworkArchitecture<T> architecture, string modelPath, WaveGlowOptions options) |
Creates a trainable WaveGlow. |
Properties
| Property | Summary |
|---|---|
int UpsampleFactor { get; } |
Methods
| Method | Summary |
|---|---|
ModelMetadata<T> GetModelMetadata() |
|
Tensor<T> Decode(Tensor<T> mel, Tensor<T> z) |
The waveform [1, 1, samples] of a latent z [1, group, samples / group] (laid out as Encode returns it) given the mel spectrogram: the exact inverse of Encode. |
ValueTuple<Tensor<T>, Tensor<T>> Encode(Tensor<T> mel, Tensor<T> audio) |
The latent z [1, group, samples / group] of a waveform [samples] given its mel spectrogram [1, mel, frames], and the log-determinant of the flow. |