SpeakerEmbeddingExtractor<T>

Models & Types · AiDotNet.Audio.Speaker

Extracts speaker embeddings (d-vectors) from audio for speaker recognition.

Inheritance: NeuralNetworkBase<T> → AudioNeuralNetworkBase<T> → SpeakerRecognitionBase<T> → SpeakerEmbeddingExtractor<T>

Implements: ISpeakerEmbeddingExtractor<T>

For Beginners

Each person’s voice has unique characteristics like pitch, rhythm, and timbre (tone color). This class converts audio into a numerical “fingerprint” of the speaker’s voice.

These embeddings are vectors (lists of numbers) that are:

  • Close together for the same speaker
  • Far apart for different speakers

Usage (ONNX Mode):

Usage (Native Training Mode):

How It Works

Speaker embeddings are compact vector representations that capture the unique characteristics of a speaker’s voice. These can be used for speaker verification (is this the same person?) and speaker identification (who is speaking?).

Constructors

Constructor Summary
SpeakerEmbeddingExtractor() Creates a SpeakerEmbeddingExtractor with default settings for native training mode.
SpeakerEmbeddingExtractor(NeuralNetworkArchitecture<T> architecture, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer, ILossFunction<T> lossFunction, SpeakerEmbeddingOptions options) Creates a SpeakerEmbeddingExtractor for native training mode.
SpeakerEmbeddingExtractor(NeuralNetworkArchitecture<T> architecture, string modelPath, SpeakerEmbeddingOptions options) Creates a SpeakerEmbeddingExtractor for ONNX inference with a pretrained model.
SpeakerEmbeddingExtractor(SpeakerEmbeddingOptions options) Creates a SpeakerEmbeddingExtractor with custom options.

Properties

Property Summary
bool HasNeuralModel { get; } Gets whether a neural model is loaded.
bool IsOnnxMode { get; } Gets whether the model is in ONNX inference mode.
double MinimumDurationSeconds { get; } Gets the minimum audio duration required for reliable embedding extraction.

Methods

Method Summary
IReadOnlyList<Tensor<T>> ExtractEmbeddings(IReadOnlyList<Tensor<T>> audioSegments) Extracts embeddings from multiple audio segments.
List<SpeakerEmbedding<T>> ExtractBatch(IEnumerable<Tensor<T>> segments) Extracts embeddings from multiple audio segments (legacy API).
ModelMetadata<T> GetModelMetadata() Gets metadata about the model.
SpeakerEmbedding<T> Extract(Tensor<T> audio) Extracts a speaker embedding from audio (legacy API).
SpeakerEmbedding<T> Extract(Vector<T> audio) Extracts a speaker embedding from audio (legacy API).
T ComputeSimilarity(SpeakerEmbedding<T> embedding1, SpeakerEmbedding<T> embedding2) Computes cosine similarity between two speaker embeddings (legacy API).
T ComputeSimilarity(Tensor<T> embedding1, Tensor<T> embedding2) Computes cosine similarity between two speaker embeddings (legacy API).
Task<Tensor<T>> ExtractEmbeddingAsync(Tensor<T> audio, CancellationToken cancellationToken) Extracts a speaker embedding from audio asynchronously.
Tensor<T> ExtractEmbedding(Tensor<T> audio) Extracts a speaker embedding from audio.
Tensor<T> ExtractTensor(Tensor<T> audio) Extracts speaker embedding from audio as a Tensor.
Tensor<T> ForwardForTraining(Tensor<T> input)