AmplitudePhaseVocoderBase<T>
Base Classes · AiDotNet.TextToSpeech.Vocoders
The frame-level amplitude-and-phase vocoders’ shared generator and losses (APNet, Ai and Ling 2023; APNet2, Du et al. 2023): an amplitude spectrum predictor (ASP) and a phase spectrum predictor (PSP) whose two parallel outputs give the phase by the two-argument arctangent, an inverse STFT, and the multilevel loss λ_A L_A + λ_P L_P + λ_S L_S + L_W.
Inheritance: NeuralNetworkBase<T> → TtsModelBase<T> → VocoderBase<T> → GanVocoderBase<T> → AmplitudePhaseVocoderBase<T>
Known derived types: APNet2<T>, APNet<T>
For Beginners
These vocoders predict how loud each frequency is and where each wave is in its cycle for every frame, then an inverse Fourier transform turns that into audio.
How It Works
log  = ASP(M), P̂ = Φ(R, I), x̂ = ISTFT( e^{jP̂}). L_A is the log-amplitude MSE; L_P the
instantaneous-phase, group-delay and phase-time-difference errors through the model’s anti-wrapping function; L_S the
STFT consistency loss between Ŝ and STFT(x̂) plus λ_RI times the L1 real and imaginary part losses; L_W the model’s
adversarial and feature-matching terms plus λ_Mel times the L1 mel loss.
Properties
| Property | Summary |
|---|---|
int UpsampleFactor { get; } |
Methods
| Method | Summary |
|---|---|
ValueTuple<Tensor<T>, Tensor<T>, Tensor<T>, Tensor<T>> AnalyzeSpectra(Tensor<T> audio) |
The natural log amplitude ln(|S| + 1e-5), phase, real and imaginary parts [1, bins, frames] of a waveform [samples] (reference amp_pha_specturm). |
ValueTuple<Tensor<T>, Tensor<T>> PredictSpectra(Tensor<T> mel) |
The log amplitude and phase spectra [1, bins, frames] the ASP and PSP predict for a mel spectrogram [1, mel, frames]; the phase is Φ(R, I), the two-argument arctangent (Eq. |