PitchWaveletTransform

Helpers & Utilities · AiDotNet.Audio.Pitch

The pitch spectrogram of FastSpeech 2: a 10-scale continuous wavelet transform of the normalized log-F0 contour, and its approximate inverse.

How It Works

FastSpeech 2 (Ren et al. 2021, §2.3 and App. C) predicts pitch as a wavelet “pitch spectrogram” rather than a raw contour: unvoiced frames are filled by linear interpolation, the contour is taken to log scale and normalized per utterance, then decomposed with the Mexican hat wavelet into 10 scales; the predicted spectrogram is recomposed with F0 = Σ W_i (i + 2.5)^(-5/2) and denormalized with the predicted utterance mean and deviation.

This follows the authors’ implementation (NATSpeech utils/audio/cwt.py, which calls pycwt): sampling interval dt = 0.005 s regardless of hop, scale spacing dj = 1, smallest scale s0 = 2 dt, J = 9 (10 scales), and pycwt’s scipy code path, which zero-pads the signal to the next power of two before transforming and trims the result back. The recomposed contour is re-standardized over time before the predicted statistics are applied, as inverse_cwt does.

Methods

Method Summary
ValueTuple<double[], double[]> ContinuousLogF0(double[] f0) ContinuousLogF0 on a tensor contour [frames] with engine operations: each frame reads the voiced frame at its position clamped to the first and last voiced frame, or interpolates linearly between the voiced frames around it.
double[] Forward(double[] signal) Forward on a tensor signal [time] with engine operations, returned as [time, scale]: one FFT of the zero-padded signal, the ten scales’ wavelet spectra as one [scale, n] product, and one batched inverse FFT.
double[] Inverse(double[] spectrogram) Recomposes a pitch spectrogram [time, scale] into a standardized log-F0 contour (NATSpeech inverse_cwt: Σ W_i (i + 2.5)^(-5/2) over i = 1..10, then zero mean and unit deviation over time).
double[] ToF0(double[] spectrogram, double logF0Mean, double logF0Std) ToF0 on a tensor spectrogram [time, scale]: the weighted scale sum, standardized over time, then exp(z σ + μ), [time].

Fields

Field Summary
double SamplingInterval Wavelet sampling interval used by the reference implementation, in seconds.
int ScaleCount Number of wavelet scales in the pitch spectrogram.