PriorGrad<T>
Models & Types · AiDotNet.TextToSpeech.Vocoders
PriorGrad vocoder: DiffWave whose diffusion prior is a data-dependent diagonal Gaussian N(0, Σ_c) with the standard deviation of each frame taken from the mel spectrogram’s normalized frame energy.
Inheritance: NeuralNetworkBase<T> → TtsModelBase<T> → VocoderBase<T> → SegmentVocoderBase<T> → DiffusionVocoderBase<T> → PriorGrad<T>
For Beginners
DiffWave starts every synthesis from the same plain noise; PriorGrad starts from noise that is already loud where the speech is loud and quiet where it is quiet, so there is less left to learn.
How It Works
References: “PriorGrad: Improving Conditional Denoising Diffusion Models with Data-Dependent Adaptive Prior” (Lee et al., ICLR 2022) and microsoft/NeuralSpeech PriorGrad-vocoder for what the paper leaves unstated.
Training (Algorithm 1): ε ~ N(0, Σ), t uniform, x_t = √ᾱ_t x₀ + √(1 − ᾱ_t) ε and the Mahalanobis loss
‖ε − ε_θ(x_t, c, t)‖²_{Σ⁻¹}. Sampling (Algorithm 2): x_T ~ N(0, Σ) and every step’s noise z ~ N(0, Σ). Σ is
the frame energy √Σ_bands exp(mel) normalized to (0, 1], clipped below at 0.1 and repeated over each frame’s hop
(§4). The network is DiffWave’s, unchanged.
Constructors
| Constructor | Summary |
|---|---|
PriorGrad(NeuralNetworkArchitecture<T> architecture, PriorGradOptions options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer) |
Creates a trainable PriorGrad. |
PriorGrad(NeuralNetworkArchitecture<T> architecture, string modelPath, PriorGradOptions options) |
Creates a trainable PriorGrad. |
Properties
| Property | Summary |
|---|---|
int UpsampleFactor { get; } |
Methods
| Method | Summary |
|---|---|
ModelMetadata<T> GetModelMetadata() |
|
Tensor<T> PriorStd(Tensor<T> mel) |
The prior’s standard deviation per sample [1, 1, samples] for a mel spectrogram [1, mel, frames]: each frame’s energy normalized to (0, 1], clipped below at the minimum standard deviation, repeated over the hop. |
void FitEnergyStatistics(IEnumerable<Tensor<T>> recordings, bool useDataMaximum) |
Sets EnergyMin (and, with useDataMaximum, EnergyMax) to the extremes of the recordings’ frame energies, as the reference computes them over the training set. |