TFGridNet<T>

Models & Types · AiDotNet.Audio.Enhancement

TF-GridNet (Time-Frequency GridNet) for speech enhancement and separation.

Inheritance: NeuralNetworkBase<T> → AudioNeuralNetworkBase<T> → TFGridNet<T>

Implements: IAudioEnhancer<T>, IGeneratedParameterRegistrar<T>

For Beginners

TF-GridNet cleans audio by processing a time-frequency grid in two alternating directions: across frequencies (understanding harmonic structure) and across time (tracking how sounds evolve). This grid approach captures both local and global patterns.

Usage:

How It Works

TF-GridNet (Wang et al., ICASSP 2023) applies alternating attention along the time and frequency axes in a grid pattern, achieving 23.4 dB SI-SNRi on WSJ0-2mix and PESQ 3.41 on DNS Challenge.

Architecture:

  • Input embedding: Maps complex STFT to per-bin embeddings
  • Grid blocks: Each block has intra-frame (frequency) and inter-frame (time) modules
  • Intra-frame: LSTM/attention across frequency bins for each time frame
  • Inter-frame: LSTM/attention across time frames for each frequency bin
  • Output: Reconstructs complex STFT for synthesis

References:

  • Paper: “TF-GridNet: Making Time-Frequency Domain Models Great Again” (Wang et al., ICASSP 2023)
  • Repository: https://github.com/espnet/espnet

Constructors

Constructor Summary
TFGridNet(NeuralNetworkArchitecture<T> architecture, TFGridNetOptions options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer) Creates a TF-GridNet model in native training mode.
TFGridNet(NeuralNetworkArchitecture<T> architecture, string modelPath, TFGridNetOptions options) Creates a TF-GridNet model in native training mode.

Properties

Property Summary
double EnhancementStrength { get; set; }
int LatencySamples { get; }
int NumChannels { get; }

Methods

Method Summary
ModelMetadata<T> GetModelMetadata()
Tensor<T> Enhance(Tensor<T> audio)
Tensor<T> EnhanceWithReference(Tensor<T> audio, Tensor<T> reference)
Tensor<T> ForwardForTraining(Tensor<T> input) Mirrors PredictCore’s utterance-level RMS scale normalization on the TRAINING forward path so training and inference see the same preprocessing.
Tensor<T> ProcessChunk(Tensor<T> audioChunk)
void EstimateNoiseProfile(Tensor<T> noiseOnlyAudio)
void ResetState()