Wav2Vec2LanguageIdentifier<T>
Models & Types · AiDotNet.Audio.LanguageIdentification
Wav2Vec2 model fine-tuned for spoken language identification.
Inheritance: NeuralNetworkBase<T> → AudioNeuralNetworkBase<T> → Wav2Vec2LanguageIdentifier<T>
Implements: ILanguageIdentifier<T>
For Beginners
Wav2Vec2 is like a very attentive listener that:
- First breaks down the raw sound wave into small pieces (feature encoder)
- Then looks at how all these pieces relate to each other (transformer)
- Finally makes a decision about what language is being spoken (classifier)
Key advantages:
- Works directly on raw audio (no need for handcrafted features like MFCCs)
- Pre-trained on massive amounts of unlabeled speech data
- Can recognize languages even with limited labeled training data
Example usage:
How It Works
Wav2Vec2 is Meta’s self-supervised speech representation learning model that learns powerful representations directly from raw audio waveforms. When fine-tuned for language identification, it achieves state-of-the-art performance on many benchmarks.
Architecture overview:
- Feature Encoder: 7 temporal convolution layers that process raw waveform
- Transformer Encoder: 12-24 transformer blocks for contextual representations
- Classification Head: Linear projection to language classes
Constructors
| Constructor | Summary |
|---|---|
Wav2Vec2LanguageIdentifier(NeuralNetworkArchitecture<T> architecture, IReadOnlyList<string> supportedLanguages, Wav2Vec2LidOptions options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer, ILossFunction<T> lossFunction) |
Creates a Wav2Vec2 language identifier for native training. |
Wav2Vec2LanguageIdentifier(NeuralNetworkArchitecture<T> architecture, string modelPath, Wav2Vec2LidOptions options) |
Creates a Wav2Vec2 language identifier with ONNX model for inference. |
Properties
| Property | Summary |
|---|---|
IReadOnlyList<string> SupportedLanguages { get; } |
|
bool SupportsTraining { get; } |
|
int HiddenSize { get; } |
Gets the hidden size of the transformer. |
Methods
| Method | Summary |
|---|---|
Dictionary<string, Tensor<T>> GetNamedLayerActivations(Tensor<T> input) |
|
IReadOnlyDictionary<string, T> GetLanguageProbabilities(Tensor<T> audio) |
|
IReadOnlyList<LanguageSegment<T>> IdentifyLanguageSegments(Tensor<T> audio, int windowSizeMs) |
|
IReadOnlyList<ValueTuple<string, T>> GetTopLanguages(Tensor<T> audio, int topN) |
|
LanguageResult<T> IdentifyLanguage(Tensor<T> audio) |
|
ModelMetadata<T> GetModelMetadata() |
|
Tensor<T> ForwardForTraining(Tensor<T> input) |
|
ValueTuple<bool, T> AreSameLanguage(Tensor<T> audio1, Tensor<T> audio2) |
|
string GetLanguageDisplayName(string languageCode) |
|
void LoadTorchWeights(Func<string, int[], double[]> read) |
Loads a transformers Wav2Vec2ForSequenceClassification state dict straight from source. |