WavLMSpeaker<T>

Models & Types · AiDotNet.Audio.Speaker

WavLM-based speaker verification and embedding extraction model (Chen et al., 2022).

Inheritance: NeuralNetworkBase<T> → AudioNeuralNetworkBase<T> → SpeakerRecognitionBase<T> → WavLMSpeaker<T>

Implements: ISpeakerEmbeddingExtractor<T>, ISpeakerVerifier<T>

For Beginners

WavLM first learned to understand speech generally by “listening” to thousands of hours of audio. Then it was specialized to recognize individual voices. Because of this broad training, it handles noisy phone calls, different microphones, and room echoes much better than models trained only on clean speech.

Usage:

How It Works

WavLM is a self-supervised speech model that, when fine-tuned for speaker verification, achieves 0.59% EER on VoxCeleb1—among the best results for any single model. Its Transformer encoder is pre-trained with masked speech prediction and denoising, making it robust to noise and reverberation.

Constructors

Constructor Summary
WavLMSpeaker(NeuralNetworkArchitecture<T> architecture, WavLMSpeakerOptions options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer)
WavLMSpeaker(NeuralNetworkArchitecture<T> architecture, string modelPath, WavLMSpeakerOptions options)

Properties

Property Summary
ISpeakerEmbeddingExtractor<T> EmbeddingExtractor { get; }
T DefaultThreshold { get; }
bool IsOnnxMode { get; }
double MinimumDurationSeconds { get; }
double VerificationThreshold { get; }

Methods

Method Summary
IReadOnlyList<Tensor<T>> ExtractEmbeddings(IReadOnlyList<Tensor<T>> audioSegments)
ModelMetadata<T> GetModelMetadata()
SpeakerProfile<T> Enroll(IReadOnlyList<Tensor<T>> enrollmentAudio)
SpeakerProfile<T> Enroll(Tensor<T> enrollmentAudio)
SpeakerProfile<T> UpdateProfile(SpeakerProfile<T> existingProfile, Tensor<T> newAudio)
SpeakerVerificationResult<T> Verify(Tensor<T> audio, Tensor<T> referenceEmbedding)
SpeakerVerificationResult<T> Verify(Tensor<T> audio, Tensor<T> referenceEmbedding, T threshold)
SpeakerVerificationResult<T> VerifyWithReferenceAudio(Tensor<T> audio, Tensor<T> referenceAudio)
T ComputeScore(Tensor<T> audio, Tensor<T> referenceEmbedding)
T ComputeSimilarity(Tensor<T> embedding1, Tensor<T> embedding2)
T GetThresholdForFAR(double targetFAR)
Task<SpeakerVerificationResult<T>> VerifyAsync(Tensor<T> audio, Tensor<T> referenceEmbedding, CancellationToken cancellationToken)
Task<Tensor<T>> ExtractEmbeddingAsync(Tensor<T> audio, CancellationToken cancellationToken)
Tensor<T> ExtractEmbedding(Tensor<T> audio)