CMGAN<T>

Models & Types · AiDotNet.Audio.Enhancement

CMGAN (Conformer-based Metric GAN) for speech enhancement.

Inheritance: NeuralNetworkBase<T> → AudioNeuralNetworkBase<T> → CMGAN<T>

Implements: IAudioEnhancer<T>, IGeneratedParameterRegistrar<T>

For Beginners

CMGAN uses a competition between two networks: a “generator” that cleans audio and a “discriminator” that judges quality. The generator uses Conformer layers that combine attention (understanding context) with convolution (local patterns).

Usage:

How It Works

CMGAN (Cao et al., INTERSPEECH 2022) combines a conformer-based generator with a metric discriminator for high-quality speech enhancement, achieving PESQ 3.41 and STOI 0.97 on the VoiceBank-DEMAND dataset.

Architecture:

  • U-Net encoder: Compresses the noisy spectrogram with convolutional blocks
  • Conformer bottleneck: Self-attention + convolution for global context
  • U-Net decoder: Reconstructs clean spectrogram with skip connections
  • Metric discriminator: Judges enhancement quality during training

References:

  • Paper: “CMGAN: Conformer-based Metric GAN for Speech Enhancement” (Cao et al., INTERSPEECH 2022)
  • Repository: https://github.com/ruizhecao96/CMGAN

Constructors

Constructor Summary
CMGAN(NeuralNetworkArchitecture<T> architecture, CMGANOptions options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer) Creates a CMGAN model in native training mode.
CMGAN(NeuralNetworkArchitecture<T> architecture, string modelPath, CMGANOptions options) Creates a CMGAN model in native training mode.

Properties

Property Summary
double EnhancementStrength { get; set; }
int LatencySamples { get; }
int NumChannels { get; }

Methods

Method Summary
ModelMetadata<T> GetModelMetadata()
Tensor<T> Enhance(Tensor<T> audio)
Tensor<T> EnhanceWithReference(Tensor<T> audio, Tensor<T> reference)
Tensor<T> ProcessChunk(Tensor<T> audioChunk)
void EstimateNoiseProfile(Tensor<T> noiseOnlyAudio)
void ResetState()