ECAPATDNNLanguageIdentifier<T>

Models & Types · AiDotNet.Audio.LanguageIdentification

ECAPA-TDNN (Emphasized Channel Attention, Propagation and Aggregation Time Delay Neural Network) for spoken language identification.

Inheritance: NeuralNetworkBase<T> → AudioNeuralNetworkBase<T> → ECAPATDNNLanguageIdentifier<T>

Implements: ILanguageIdentifier<T>

For Beginners

ECAPA-TDNN is like having a very sophisticated listener that can:

  1. Hear patterns at different time scales (TDNN layers)
  2. Focus on the most important sound characteristics (channel attention)
  3. Combine information from multiple processing stages (MFA)
  4. Handle audio of any length (attentive pooling)

This model is particularly good at:

  • Identifying languages from short audio clips (3-10 seconds)
  • Handling noisy or low-quality audio
  • Distinguishing between similar languages (e.g., Spanish vs Portuguese)

Example usage:

How It Works

ECAPA-TDNN is a state-of-the-art architecture originally designed for speaker verification that has been adapted for language identification. It uses:

  • Time Delay Neural Network (TDNN) layers with dilated convolutions
  • Squeeze-Excitation (SE) blocks for channel attention
  • Multi-layer feature aggregation (MFA) for combining information across layers
  • Attentive statistics pooling for variable-length utterances

Constructors

Constructor Summary
ECAPATDNNLanguageIdentifier(NeuralNetworkArchitecture<T> architecture, IReadOnlyList<string> supportedLanguages, ECAPATDNNOptions options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer, ILossFunction<T> lossFunction) Creates an ECAPA-TDNN language identifier for native training.
ECAPATDNNLanguageIdentifier(NeuralNetworkArchitecture<T> architecture, string modelPath, ECAPATDNNOptions options) Creates an ECAPA-TDNN language identifier with ONNX model for inference.

Properties

Property Summary
IReadOnlyList<string> SupportedLanguages { get; }
bool SupportsTraining { get; }
int EmbeddingDimension { get; } Gets the embedding dimension produced by this model.

Methods

Method Summary
IReadOnlyDictionary<string, T> GetLanguageProbabilities(Tensor<T> audio)
IReadOnlyList<LanguageSegment<T>> IdentifyLanguageSegments(Tensor<T> audio, int windowSizeMs)
IReadOnlyList<ValueTuple<string, T>> GetTopLanguages(Tensor<T> audio, int topN)
LanguageResult<T> IdentifyLanguage(Tensor<T> audio)
ModelMetadata<T> GetModelMetadata()
Tensor<T> ForwardForTraining(Tensor<T> input)
ValueTuple<bool, T> AreSameLanguage(Tensor<T> audio1, Tensor<T> audio2)
string GetLanguageDisplayName(string languageCode)