CMGAN<T>
Models & Types · AiDotNet.Audio.Enhancement
CMGAN (Conformer-based Metric GAN) for speech enhancement.
Inheritance: NeuralNetworkBase<T> → AudioNeuralNetworkBase<T> → CMGAN<T>
Implements: IAudioEnhancer<T>, IGeneratedParameterRegistrar<T>
For Beginners
CMGAN uses a competition between two networks: a “generator” that cleans audio and a “discriminator” that judges quality. The generator uses Conformer layers that combine attention (understanding context) with convolution (local patterns).
Usage:
How It Works
CMGAN (Cao et al., INTERSPEECH 2022) combines a conformer-based generator with a metric discriminator for high-quality speech enhancement, achieving PESQ 3.41 and STOI 0.97 on the VoiceBank-DEMAND dataset.
Architecture:
- U-Net encoder: Compresses the noisy spectrogram with convolutional blocks
- Conformer bottleneck: Self-attention + convolution for global context
- U-Net decoder: Reconstructs clean spectrogram with skip connections
- Metric discriminator: Judges enhancement quality during training
References:
- Paper: “CMGAN: Conformer-based Metric GAN for Speech Enhancement” (Cao et al., INTERSPEECH 2022)
- Repository:
https://github.com/ruizhecao96/CMGAN
Constructors
| Constructor | Summary |
|---|---|
CMGAN(NeuralNetworkArchitecture<T> architecture, CMGANOptions options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer) |
Creates a CMGAN model in native training mode. |
CMGAN(NeuralNetworkArchitecture<T> architecture, string modelPath, CMGANOptions options) |
Creates a CMGAN model in native training mode. |
Properties
| Property | Summary |
|---|---|
double EnhancementStrength { get; set; } |
|
int LatencySamples { get; } |
|
int NumChannels { get; } |
Methods
| Method | Summary |
|---|---|
ModelMetadata<T> GetModelMetadata() |
|
Tensor<T> Enhance(Tensor<T> audio) |
|
Tensor<T> EnhanceWithReference(Tensor<T> audio, Tensor<T> reference) |
|
Tensor<T> ProcessChunk(Tensor<T> audioChunk) |
|
void EstimateNoiseProfile(Tensor<T> noiseOnlyAudio) |
|
void ResetState() |