ISpeakerDiarizer<T>
Interfaces · AiDotNet.Interfaces
Interface for speaker diarization models that segment audio by speaker (“who spoke when”).
Implements: IFeatureAware, IFullModel<T, Tensor<T>, Tensor<T>>, IGradientComputable<T, Tensor<T>, Tensor<T>>
Known implementations: PyAnnote<T>, SpeakerDiarizer<T>
For Beginners
Diarization is like labeling a transcript with “Speaker A said…” “Speaker B said…” without knowing their names.
How it works:
- Audio is segmented into small chunks
- Speaker embeddings are extracted for each chunk
- Clustering groups similar embeddings together
- Each cluster represents a unique speaker
- Output: Timeline showing when each speaker talks
Common use cases:
- Meeting transcription (separating participants)
- Podcast/interview processing
- Call center analytics
- Medical dictation
Challenges:
- Overlapping speech (multiple people talking at once)
- Short turns (quick back-and-forth conversation)
- Similar voices (e.g., siblings)
- Background noise and music
How It Works
Speaker diarization partitions an audio stream into segments based on speaker identity. It answers the question “Who spoke when?” without necessarily knowing who the speakers are (unlike speaker identification which requires enrolled speakers).
This interface extends IFullModel for Tensor-based audio processing.
Properties
| Property | Summary |
|---|---|
bool IsOnnxMode { get; } |
Gets whether this model is running in ONNX inference mode. |
bool SupportsOverlapDetection { get; } |
Gets whether this model can detect overlapping speech. |
double MinSegmentDuration { get; } |
Gets the minimum segment duration in seconds. |
int SampleRate { get; } |
Gets the expected sample rate for input audio. |
Methods
| Method | Summary |
|---|---|
DiarizationResult<T> Diarize(Tensor<T> audio, int? numSpeakers, int minSpeakers, int maxSpeakers) |
Performs speaker diarization on audio. |
DiarizationResult<T> DiarizeWithKnownSpeakers(Tensor<T> audio, IReadOnlyList<SpeakerProfile<T>> knownSpeakers, bool allowUnknownSpeakers) |
Performs diarization with known speaker profiles. |
DiarizationResult<T> RefineDiarization(Tensor<T> audio, DiarizationResult<T> previousResult, T mergeThreshold) |
Refines diarization result by re-segmenting with different parameters. |
IReadOnlyDictionary<string, Tensor<T>> ExtractSpeakerEmbeddings(Tensor<T> audio, DiarizationResult<T> diarizationResult) |
Gets speaker embeddings for each detected speaker. |
Task<DiarizationResult<T>> DiarizeAsync(Tensor<T> audio, int? numSpeakers, int minSpeakers, int maxSpeakers, CancellationToken cancellationToken) |
Performs speaker diarization asynchronously. |