WavLMSpeaker<T>
Models & Types · AiDotNet.Audio.Speaker
WavLM-based speaker verification and embedding extraction model (Chen et al., 2022).
Inheritance: NeuralNetworkBase<T> → AudioNeuralNetworkBase<T> → SpeakerRecognitionBase<T> → WavLMSpeaker<T>
Implements: ISpeakerEmbeddingExtractor<T>, ISpeakerVerifier<T>
For Beginners
WavLM first learned to understand speech generally by “listening” to thousands of hours of audio. Then it was specialized to recognize individual voices. Because of this broad training, it handles noisy phone calls, different microphones, and room echoes much better than models trained only on clean speech.
Usage:
How It Works
WavLM is a self-supervised speech model that, when fine-tuned for speaker verification, achieves 0.59% EER on VoxCeleb1—among the best results for any single model. Its Transformer encoder is pre-trained with masked speech prediction and denoising, making it robust to noise and reverberation.
Constructors
| Constructor | Summary |
|---|---|
WavLMSpeaker(NeuralNetworkArchitecture<T> architecture, WavLMSpeakerOptions options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer) |
|
WavLMSpeaker(NeuralNetworkArchitecture<T> architecture, string modelPath, WavLMSpeakerOptions options) |
Properties
| Property | Summary |
|---|---|
ISpeakerEmbeddingExtractor<T> EmbeddingExtractor { get; } |
|
T DefaultThreshold { get; } |
|
bool IsOnnxMode { get; } |
|
double MinimumDurationSeconds { get; } |
|
double VerificationThreshold { get; } |
Methods
| Method | Summary |
|---|---|
IReadOnlyList<Tensor<T>> ExtractEmbeddings(IReadOnlyList<Tensor<T>> audioSegments) |
|
ModelMetadata<T> GetModelMetadata() |
|
SpeakerProfile<T> Enroll(IReadOnlyList<Tensor<T>> enrollmentAudio) |
|
SpeakerProfile<T> Enroll(Tensor<T> enrollmentAudio) |
|
SpeakerProfile<T> UpdateProfile(SpeakerProfile<T> existingProfile, Tensor<T> newAudio) |
|
SpeakerVerificationResult<T> Verify(Tensor<T> audio, Tensor<T> referenceEmbedding) |
|
SpeakerVerificationResult<T> Verify(Tensor<T> audio, Tensor<T> referenceEmbedding, T threshold) |
|
SpeakerVerificationResult<T> VerifyWithReferenceAudio(Tensor<T> audio, Tensor<T> referenceAudio) |
|
T ComputeScore(Tensor<T> audio, Tensor<T> referenceEmbedding) |
|
T ComputeSimilarity(Tensor<T> embedding1, Tensor<T> embedding2) |
|
T GetThresholdForFAR(double targetFAR) |
|
Task<SpeakerVerificationResult<T>> VerifyAsync(Tensor<T> audio, Tensor<T> referenceEmbedding, CancellationToken cancellationToken) |
|
Task<Tensor<T>> ExtractEmbeddingAsync(Tensor<T> audio, CancellationToken cancellationToken) |
|
Tensor<T> ExtractEmbedding(Tensor<T> audio) |