ISpeakerEmbeddingExtractor<T>
Interfaces · AiDotNet.Interfaces
Interface for speaker embedding extraction models (d-vector/x-vector extraction).
Implements: IFullModel<T, Tensor<T>, Tensor<T>>
Known implementations: CAMPlusPlus<T>, ECAPATDNNSpeaker<T>, SpeakerEmbeddingExtractor<T>, SpeakerLM<T>, TitaNet<T>, WavLMSpeaker<T>
For Beginners
Speaker embeddings are like a “voiceprint” - a compact representation of what makes someone’s voice unique.
How speaker embeddings work:
- Audio of someone speaking is fed into the model
- The model outputs a fixed-size vector (e.g., 256 or 512 numbers)
- This vector captures voice characteristics (pitch, timbre, accent, etc.)
- Vectors from the same speaker are similar; different speakers are different
Common use cases:
- Voice authentication (“Is this person who they claim to be?”)
- Speaker identification (“Who is speaking?”)
- Voice cloning (TTS with specific voice)
- Meeting transcription (separating speakers)
Key concepts:
- d-vector: Early embedding approach using DNN
- x-vector: Modern approach using TDNN with statistics pooling
- ECAPA-TDNN: State-of-the-art speaker embedding model
How It Works
Speaker embedding extractors convert voice audio into fixed-length vectors that capture the unique characteristics of a speaker’s voice. These embeddings enable speaker verification, identification, and diarization tasks.
This interface extends IFullModel for Tensor-based audio processing.
Properties
| Property | Summary |
|---|---|
bool IsOnnxMode { get; } |
Gets whether this model is running in ONNX inference mode. |
double MinimumDurationSeconds { get; } |
Gets the minimum audio duration required for reliable embedding extraction. |
int EmbeddingDimension { get; } |
Gets the dimension of output speaker embeddings. |
int SampleRate { get; } |
Gets the expected sample rate for input audio. |
Methods
| Method | Summary |
|---|---|
IReadOnlyList<Tensor<T>> ExtractEmbeddings(IReadOnlyList<Tensor<T>> audioSegments) |
Extracts embeddings from multiple audio segments. |
T ComputeSimilarity(Tensor<T> embedding1, Tensor<T> embedding2) |
Computes similarity between two speaker embeddings. |
Task<Tensor<T>> ExtractEmbeddingAsync(Tensor<T> audio, CancellationToken cancellationToken) |
Extracts speaker embedding from audio asynchronously. |
Tensor<T> AggregateEmbeddings(IReadOnlyList<Tensor<T>> embeddings) |
Aggregates multiple embeddings into a single representative embedding. |
Tensor<T> ExtractEmbedding(Tensor<T> audio) |
Extracts speaker embedding from audio. |
Tensor<T> NormalizeEmbedding(Tensor<T> embedding) |
Normalizes an embedding for comparison (typically L2 normalization). |