SpeakerLM<T>
Models & Types · AiDotNet.Audio.Speaker
SpeakerLM language-model-based speaker diarization and verification model.
Inheritance: NeuralNetworkBase<T> → AudioNeuralNetworkBase<T> → SpeakerRecognitionBase<T> → SpeakerLM<T>
Implements: ISpeakerEmbeddingExtractor<T>, ISpeakerVerifier<T>
For Beginners
SpeakerLM figures out “who said what” in a conversation by treating speaker changes like a language. Just as a language model predicts the next word, SpeakerLM predicts the next speaker turn. It learns patterns like “after person A speaks, person B usually responds” to improve accuracy.
Usage:
How It Works
SpeakerLM (2024) applies language modeling techniques to speaker embeddings for improved speaker diarization. It treats speaker turns as a sequence modeling problem, using a transformer-based language model over speaker embeddings to predict who speaks when, achieving improved DER on common benchmarks.
Constructors
| Constructor | Summary |
|---|---|
SpeakerLM(NeuralNetworkArchitecture<T> architecture, SpeakerLMOptions options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer) |
Creates a SpeakerLM model in native training mode. |
SpeakerLM(NeuralNetworkArchitecture<T> architecture, string modelPath, SpeakerLMOptions options) |
Creates a SpeakerLM model in native training mode. |
Properties
| Property | Summary |
|---|---|
ISpeakerEmbeddingExtractor<T> EmbeddingExtractor { get; } |
|
T DefaultThreshold { get; } |
|
bool IsOnnxMode { get; } |
|
double MinimumDurationSeconds { get; } |
Methods
| Method | Summary |
|---|---|
IReadOnlyList<Tensor<T>> ExtractEmbeddings(IReadOnlyList<Tensor<T>> audioSegments) |
|
ModelMetadata<T> GetModelMetadata() |
|
SpeakerProfile<T> Enroll(IReadOnlyList<Tensor<T>> enrollmentAudio) |
|
SpeakerProfile<T> Enroll(Tensor<T> enrollmentAudio) |
|
SpeakerProfile<T> UpdateProfile(SpeakerProfile<T> existingProfile, Tensor<T> newAudio) |
|
SpeakerVerificationResult<T> Verify(Tensor<T> audio, Tensor<T> referenceEmbedding) |
|
SpeakerVerificationResult<T> Verify(Tensor<T> audio, Tensor<T> referenceEmbedding, T threshold) |
|
SpeakerVerificationResult<T> VerifyWithReferenceAudio(Tensor<T> audio, Tensor<T> referenceAudio) |
|
T ComputeScore(Tensor<T> audio, Tensor<T> referenceEmbedding) |
|
T ComputeSimilarity(Tensor<T> embedding1, Tensor<T> embedding2) |
|
T GetThresholdForFAR(double targetFAR) |
|
Task<SpeakerVerificationResult<T>> VerifyAsync(Tensor<T> audio, Tensor<T> referenceEmbedding, CancellationToken cancellationToken) |
|
Task<Tensor<T>> ExtractEmbeddingAsync(Tensor<T> audio, CancellationToken cancellationToken) |
|
Tensor<T> ExtractEmbedding(Tensor<T> audio) |