AmplitudePhaseVocoderBase<T>

Base Classes · AiDotNet.TextToSpeech.Vocoders

The frame-level amplitude-and-phase vocoders’ shared generator and losses (APNet, Ai and Ling 2023; APNet2, Du et al. 2023): an amplitude spectrum predictor (ASP) and a phase spectrum predictor (PSP) whose two parallel outputs give the phase by the two-argument arctangent, an inverse STFT, and the multilevel loss λ_A L_A + λ_P L_P + λ_S L_S + L_W.

Inheritance: NeuralNetworkBase<T> → TtsModelBase<T> → VocoderBase<T> → GanVocoderBase<T> → AmplitudePhaseVocoderBase<T>

Known derived types: APNet2<T>, APNet<T>

For Beginners

These vocoders predict how loud each frequency is and where each wave is in its cycle for every frame, then an inverse Fourier transform turns that into audio.

How It Works

log  = ASP(M), P̂ = Φ(R, I), x̂ = ISTFT( e^{jP̂}). L_A is the log-amplitude MSE; L_P the instantaneous-phase, group-delay and phase-time-difference errors through the model’s anti-wrapping function; L_S the STFT consistency loss between Ŝ and STFT(x̂) plus λ_RI times the L1 real and imaginary part losses; L_W the model’s adversarial and feature-matching terms plus λ_Mel times the L1 mel loss.

Properties

Property Summary
int UpsampleFactor { get; }

Methods

Method Summary
ValueTuple<Tensor<T>, Tensor<T>, Tensor<T>, Tensor<T>> AnalyzeSpectra(Tensor<T> audio) The natural log amplitude ln(|S| + 1e-5), phase, real and imaginary parts [1, bins, frames] of a waveform [samples] (reference amp_pha_specturm).
ValueTuple<Tensor<T>, Tensor<T>> PredictSpectra(Tensor<T> mel) The log amplitude and phase spectra [1, bins, frames] the ASP and PSP predict for a mel spectrogram [1, mel, frames]; the phase is Φ(R, I), the two-argument arctangent (Eq.