ECAPATDNNSpeaker<T>

Models & Types · AiDotNet.Audio.Speaker

ECAPA-TDNN speaker verification and embedding extraction model.

Inheritance: NeuralNetworkBase<T> → AudioNeuralNetworkBase<T> → SpeakerRecognitionBase<T> → ECAPATDNNSpeaker<T>

Implements: ISpeakerEmbeddingExtractor<T>, ISpeakerVerifier<T>

For Beginners

ECAPA-TDNN creates a unique “voiceprint” for any speaker. Feed it audio of someone speaking, and it returns a compact vector (embedding) that captures the speaker’s unique voice characteristics. You can compare two embeddings to check if they are the same person (verification) or find the closest match among enrolled speakers (identification).

Usage:

How It Works

ECAPA-TDNN (Desplanques et al., Interspeech 2020) is a state-of-the-art speaker embedding architecture that extends x-vectors with Squeeze-Excitation blocks, multi-layer feature aggregation, and channel- and context-dependent statistics pooling. Achieves 0.87% EER on VoxCeleb1 test set.

Constructors

Constructor Summary
ECAPATDNNSpeaker(NeuralNetworkArchitecture<T> architecture, ECAPATDNNSpeakerOptions options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer) Creates an ECAPA-TDNN speaker model in native training mode.
ECAPATDNNSpeaker(NeuralNetworkArchitecture<T> architecture, string modelPath, ECAPATDNNSpeakerOptions options) Creates an ECAPA-TDNN speaker model in native training mode.

Properties

Property Summary
ISpeakerEmbeddingExtractor<T> EmbeddingExtractor { get; }
T DefaultThreshold { get; }
bool IsOnnxMode { get; }
double MinimumDurationSeconds { get; }
double VerificationThreshold { get; }

Methods

Method Summary
IReadOnlyList<Tensor<T>> ExtractEmbeddings(IReadOnlyList<Tensor<T>> audioSegments)
ModelMetadata<T> GetModelMetadata()
SpeakerProfile<T> Enroll(IReadOnlyList<Tensor<T>> enrollmentAudio)
SpeakerProfile<T> Enroll(Tensor<T> enrollmentAudio)
SpeakerProfile<T> UpdateProfile(SpeakerProfile<T> existingProfile, Tensor<T> newAudio)
SpeakerVerificationResult<T> Verify(Tensor<T> audio, Tensor<T> referenceEmbedding)
SpeakerVerificationResult<T> Verify(Tensor<T> audio, Tensor<T> referenceEmbedding, T threshold)
SpeakerVerificationResult<T> VerifyWithReferenceAudio(Tensor<T> audio, Tensor<T> referenceAudio)
T ComputeScore(Tensor<T> audio, Tensor<T> referenceEmbedding)
T ComputeSimilarity(Tensor<T> embedding1, Tensor<T> embedding2)
T GetThresholdForFAR(double targetFAR)
Task<SpeakerVerificationResult<T>> VerifyAsync(Tensor<T> audio, Tensor<T> referenceEmbedding, CancellationToken cancellationToken)
Task<Tensor<T>> ExtractEmbeddingAsync(Tensor<T> audio, CancellationToken cancellationToken)
Tensor<T> ExtractEmbedding(Tensor<T> audio)
Tensor<T> ForwardForTraining(Tensor<T> input)