SpeakerEmbeddingExtractor<T>
Models & Types · AiDotNet.Audio.Speaker
Extracts speaker embeddings (d-vectors) from audio for speaker recognition.
Inheritance: NeuralNetworkBase<T> → AudioNeuralNetworkBase<T> → SpeakerRecognitionBase<T> → SpeakerEmbeddingExtractor<T>
Implements: ISpeakerEmbeddingExtractor<T>
For Beginners
Each person’s voice has unique characteristics like pitch, rhythm, and timbre (tone color). This class converts audio into a numerical “fingerprint” of the speaker’s voice.
These embeddings are vectors (lists of numbers) that are:
- Close together for the same speaker
- Far apart for different speakers
Usage (ONNX Mode):
Usage (Native Training Mode):
How It Works
Speaker embeddings are compact vector representations that capture the unique characteristics of a speaker’s voice. These can be used for speaker verification (is this the same person?) and speaker identification (who is speaking?).
Constructors
| Constructor | Summary |
|---|---|
SpeakerEmbeddingExtractor() |
Creates a SpeakerEmbeddingExtractor with default settings for native training mode. |
SpeakerEmbeddingExtractor(NeuralNetworkArchitecture<T> architecture, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer, ILossFunction<T> lossFunction, SpeakerEmbeddingOptions options) |
Creates a SpeakerEmbeddingExtractor for native training mode. |
SpeakerEmbeddingExtractor(NeuralNetworkArchitecture<T> architecture, string modelPath, SpeakerEmbeddingOptions options) |
Creates a SpeakerEmbeddingExtractor for ONNX inference with a pretrained model. |
SpeakerEmbeddingExtractor(SpeakerEmbeddingOptions options) |
Creates a SpeakerEmbeddingExtractor with custom options. |
Properties
| Property | Summary |
|---|---|
bool HasNeuralModel { get; } |
Gets whether a neural model is loaded. |
bool IsOnnxMode { get; } |
Gets whether the model is in ONNX inference mode. |
double MinimumDurationSeconds { get; } |
Gets the minimum audio duration required for reliable embedding extraction. |
Methods
| Method | Summary |
|---|---|
IReadOnlyList<Tensor<T>> ExtractEmbeddings(IReadOnlyList<Tensor<T>> audioSegments) |
Extracts embeddings from multiple audio segments. |
List<SpeakerEmbedding<T>> ExtractBatch(IEnumerable<Tensor<T>> segments) |
Extracts embeddings from multiple audio segments (legacy API). |
ModelMetadata<T> GetModelMetadata() |
Gets metadata about the model. |
SpeakerEmbedding<T> Extract(Tensor<T> audio) |
Extracts a speaker embedding from audio (legacy API). |
SpeakerEmbedding<T> Extract(Vector<T> audio) |
Extracts a speaker embedding from audio (legacy API). |
T ComputeSimilarity(SpeakerEmbedding<T> embedding1, SpeakerEmbedding<T> embedding2) |
Computes cosine similarity between two speaker embeddings (legacy API). |
T ComputeSimilarity(Tensor<T> embedding1, Tensor<T> embedding2) |
Computes cosine similarity between two speaker embeddings (legacy API). |
Task<Tensor<T>> ExtractEmbeddingAsync(Tensor<T> audio, CancellationToken cancellationToken) |
Extracts a speaker embedding from audio asynchronously. |
Tensor<T> ExtractEmbedding(Tensor<T> audio) |
Extracts a speaker embedding from audio. |
Tensor<T> ExtractTensor(Tensor<T> audio) |
Extracts speaker embedding from audio as a Tensor. |
Tensor<T> ForwardForTraining(Tensor<T> input) |