ShortTimeFourierTransform<T>

Models & Types · AiDotNet.Diffusion.Audio

Short-Time Fourier Transform (STFT) for analyzing audio signals over time.

For Beginners

Audio signals like music or speech change over time. While a regular FFT tells you which frequencies are in the entire signal, it doesn’t tell you WHEN those frequencies occur.

The STFT solves this by:

  1. Cutting the audio into small overlapping pieces (frames)
  2. Applying a window function to each frame (reduces edge artifacts)
  3. Computing FFT on each windowed frame
  4. Stacking the results to form a spectrogram (time vs. frequency)

Usage:

var stft = new ShortTimeFourierTransform<float>(nFft: 2048, hopLength: 512);
var audioSignal = Tensor<float>.CreateRandom(44100); // one second at 44.1 kHz
var spectrogram = stft.Forward(audioSignal);
// spectrogram.Shape = [numFrames, nFft/2 + 1] (complex values)

// To reconstruct audio from spectrogram:
var reconstructed = stft.Inverse(spectrogram);

How It Works

The STFT breaks a signal into short overlapping segments and computes the Fourier transform of each segment. This reveals how the frequency content of a signal changes over time.

Constructors

Constructor Summary
ShortTimeFourierTransform(int nFft, int? hopLength, int? windowLength, IWindowFunction<T> windowFunction, bool center, PaddingMode padMode) Initializes a new STFT processor.

Properties

Property Summary
Tensor<T> WindowTensor { get; } Gets the window tensor for GPU operations.
int HopLength { get; } Gets the hop length.
int NFft { get; } Gets the FFT size.
int NumFrequencyBins { get; } Gets the number of frequency bins (nFft / 2 + 1).

Methods

Method Summary
Tensor<Complex<T>> Forward(Tensor<T> signal) Computes the Short-Time Fourier Transform of a signal.
Tensor<Complex<T>> PolarToComplex(Tensor<T> magnitude, Tensor<T> phase) Creates complex spectrogram from magnitude and phase.
Tensor<T> ExtractPhase(Tensor<Complex<T>> complex) Extracts phase from complex spectrogram.
Tensor<T> Inverse(Tensor<Complex<T>> spectrogram, int? length) Computes the Inverse Short-Time Fourier Transform (overlap-add reconstruction).
Tensor<T> InverseFromMagnitudeAndPhase(Tensor<T> magnitude, Tensor<T> phase, int? length) Reconstructs audio signal from magnitude and phase spectrograms.
Tensor<T> Magnitude(Tensor<T> signal) Computes the magnitude spectrogram.
Tensor<T> Power(Tensor<T> signal) Computes the power spectrogram (magnitude squared).
int CalculateNumFrames(int signalLength) Calculates the number of frames for a given signal length.
int CalculateSignalLength(int numFrames) Calculates signal length from number of frames.
void MagnitudeAndPhase(Tensor<T> signal, Tensor<T> magnitude, Tensor<T> phase) Computes magnitude and phase spectrograms simultaneously.