TFGridNet<T>
Models & Types · AiDotNet.Audio.Enhancement
TF-GridNet (Time-Frequency GridNet) for speech enhancement and separation.
Inheritance: NeuralNetworkBase<T> → AudioNeuralNetworkBase<T> → TFGridNet<T>
Implements: IAudioEnhancer<T>, IGeneratedParameterRegistrar<T>
For Beginners
TF-GridNet cleans audio by processing a time-frequency grid in two alternating directions: across frequencies (understanding harmonic structure) and across time (tracking how sounds evolve). This grid approach captures both local and global patterns.
Usage:
How It Works
TF-GridNet (Wang et al., ICASSP 2023) applies alternating attention along the time and frequency axes in a grid pattern, achieving 23.4 dB SI-SNRi on WSJ0-2mix and PESQ 3.41 on DNS Challenge.
Architecture:
- Input embedding: Maps complex STFT to per-bin embeddings
- Grid blocks: Each block has intra-frame (frequency) and inter-frame (time) modules
- Intra-frame: LSTM/attention across frequency bins for each time frame
- Inter-frame: LSTM/attention across time frames for each frequency bin
- Output: Reconstructs complex STFT for synthesis
References:
- Paper: “TF-GridNet: Making Time-Frequency Domain Models Great Again” (Wang et al., ICASSP 2023)
- Repository:
https://github.com/espnet/espnet
Constructors
| Constructor | Summary |
|---|---|
TFGridNet(NeuralNetworkArchitecture<T> architecture, TFGridNetOptions options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer) |
Creates a TF-GridNet model in native training mode. |
TFGridNet(NeuralNetworkArchitecture<T> architecture, string modelPath, TFGridNetOptions options) |
Creates a TF-GridNet model in native training mode. |
Properties
| Property | Summary |
|---|---|
double EnhancementStrength { get; set; } |
|
int LatencySamples { get; } |
|
int NumChannels { get; } |
Methods
| Method | Summary |
|---|---|
ModelMetadata<T> GetModelMetadata() |
|
Tensor<T> Enhance(Tensor<T> audio) |
|
Tensor<T> EnhanceWithReference(Tensor<T> audio, Tensor<T> reference) |
|
Tensor<T> ForwardForTraining(Tensor<T> input) |
Mirrors PredictCore’s utterance-level RMS scale normalization on the TRAINING forward path so training and inference see the same preprocessing. |
Tensor<T> ProcessChunk(Tensor<T> audioChunk) |
|
void EstimateNoiseProfile(Tensor<T> noiseOnlyAudio) |
|
void ResetState() |