ISpeakerEmbeddingExtractor<T>

Interfaces · AiDotNet.Interfaces

Interface for speaker embedding extraction models (d-vector/x-vector extraction).

Implements: IFullModel<T, Tensor<T>, Tensor<T>>

Known implementations: CAMPlusPlus<T>, ECAPATDNNSpeaker<T>, SpeakerEmbeddingExtractor<T>, SpeakerLM<T>, TitaNet<T>, WavLMSpeaker<T>

For Beginners

Speaker embeddings are like a “voiceprint” - a compact representation of what makes someone’s voice unique.

How speaker embeddings work:

  1. Audio of someone speaking is fed into the model
  2. The model outputs a fixed-size vector (e.g., 256 or 512 numbers)
  3. This vector captures voice characteristics (pitch, timbre, accent, etc.)
  4. Vectors from the same speaker are similar; different speakers are different

Common use cases:

  • Voice authentication (“Is this person who they claim to be?”)
  • Speaker identification (“Who is speaking?”)
  • Voice cloning (TTS with specific voice)
  • Meeting transcription (separating speakers)

Key concepts:

  • d-vector: Early embedding approach using DNN
  • x-vector: Modern approach using TDNN with statistics pooling
  • ECAPA-TDNN: State-of-the-art speaker embedding model

How It Works

Speaker embedding extractors convert voice audio into fixed-length vectors that capture the unique characteristics of a speaker’s voice. These embeddings enable speaker verification, identification, and diarization tasks.

This interface extends IFullModel for Tensor-based audio processing.

Properties

Property Summary
bool IsOnnxMode { get; } Gets whether this model is running in ONNX inference mode.
double MinimumDurationSeconds { get; } Gets the minimum audio duration required for reliable embedding extraction.
int EmbeddingDimension { get; } Gets the dimension of output speaker embeddings.
int SampleRate { get; } Gets the expected sample rate for input audio.

Methods

Method Summary
IReadOnlyList<Tensor<T>> ExtractEmbeddings(IReadOnlyList<Tensor<T>> audioSegments) Extracts embeddings from multiple audio segments.
T ComputeSimilarity(Tensor<T> embedding1, Tensor<T> embedding2) Computes similarity between two speaker embeddings.
Task<Tensor<T>> ExtractEmbeddingAsync(Tensor<T> audio, CancellationToken cancellationToken) Extracts speaker embedding from audio asynchronously.
Tensor<T> AggregateEmbeddings(IReadOnlyList<Tensor<T>> embeddings) Aggregates multiple embeddings into a single representative embedding.
Tensor<T> ExtractEmbedding(Tensor<T> audio) Extracts speaker embedding from audio.
Tensor<T> NormalizeEmbedding(Tensor<T> embedding) Normalizes an embedding for comparison (typically L2 normalization).