SpeakerDiarizer<T>
Models & Types · AiDotNet.Audio.Speaker
Performs speaker diarization (who spoke when) on audio recordings.
Inheritance: NeuralNetworkBase<T> → AudioNeuralNetworkBase<T> → SpeakerRecognitionBase<T> → SpeakerDiarizer<T>
Implements: ISpeakerDiarizer<T>
For Beginners
Diarization is like automatically labeling a meeting recording with “Speaker A: 0:00-0:15, Speaker B: 0:15-0:45…”
The process:
- Split audio into short segments
- Extract speaker embeddings for each segment
- Cluster similar embeddings together
- Each cluster represents a different speaker
Common applications:
- Meeting transcription
- Call center analytics
- Podcast processing
Usage:
How It Works
Speaker diarization segments audio by speaker, answering “who spoke when?” It uses embeddings from sliding windows and clustering to identify speaker turns.
This class supports both:
- ONNX mode: Load pre-trained models for fast inference
- Native training mode: Train from scratch using the layer architecture
Constructors
| Constructor | Summary |
|---|---|
SpeakerDiarizer(NeuralNetworkArchitecture<T> architecture, SpeakerDiarizerOptions options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer) |
Creates a new speaker diarizer in native training mode. |
SpeakerDiarizer(NeuralNetworkArchitecture<T> architecture, string modelPath, SpeakerDiarizerOptions options) |
Creates a new speaker diarizer in native training mode. |
SpeakerDiarizer(SpeakerDiarizerOptions options) |
Creates a new speaker diarizer with legacy options only. |
Properties
| Property | Summary |
|---|---|
bool IsOnnxMode { get; } |
Gets whether the model is operating in ONNX inference mode. |
bool SupportsOverlapDetection { get; } |
Gets whether this model can detect overlapping speech. |
double ClusteringThreshold { get; } |
Gets the clustering threshold. |
double MinSegmentDuration { get; } |
Gets the minimum segment duration in seconds. |
double MinTurnDuration { get; } |
Gets the minimum turn duration in seconds. |
int SampleRate { get; } |
Gets the sample rate. |
Methods
| Method | Summary |
|---|---|
DiarizationResult DiarizeLegacy(Tensor<T> audio) |
Performs diarization on audio (legacy API). |
DiarizationResult DiarizeLegacy(Vector<T> audio) |
Performs diarization on audio (legacy API). |
DiarizationResult<T> Diarize(Tensor<T> audio, int? numSpeakers, int minSpeakers, int maxSpeakers) |
Performs speaker diarization on audio. |
DiarizationResult<T> DiarizeWithKnownSpeakers(Tensor<T> audio, IReadOnlyList<SpeakerProfile<T>> knownSpeakers, bool allowUnknownSpeakers) |
Performs diarization with known speaker profiles. |
DiarizationResult<T> RefineDiarization(Tensor<T> audio, DiarizationResult<T> previousResult, T mergeThreshold) |
Refines diarization result by re-segmenting with different parameters. |
IReadOnlyDictionary<string, Tensor<T>> ExtractSpeakerEmbeddings(Tensor<T> audio, DiarizationResult<T> diarizationResult) |
Gets speaker embeddings for each detected speaker. |
ModelMetadata<T> GetModelMetadata() |
Gets metadata about the model. |
Task<DiarizationResult<T>> DiarizeAsync(Tensor<T> audio, int? numSpeakers, int minSpeakers, int maxSpeakers, CancellationToken cancellationToken) |
Performs speaker diarization asynchronously. |
Tensor<T> ForwardForTraining(Tensor<T> input) |
|
void Dispose() |
Disposes resources. |