SpeakerDiarizer<T>

Models & Types · AiDotNet.Audio.Speaker

Performs speaker diarization (who spoke when) on audio recordings.

Inheritance: NeuralNetworkBase<T> → AudioNeuralNetworkBase<T> → SpeakerRecognitionBase<T> → SpeakerDiarizer<T>

Implements: ISpeakerDiarizer<T>

For Beginners

Diarization is like automatically labeling a meeting recording with “Speaker A: 0:00-0:15, Speaker B: 0:15-0:45…”

The process:

  1. Split audio into short segments
  2. Extract speaker embeddings for each segment
  3. Cluster similar embeddings together
  4. Each cluster represents a different speaker

Common applications:

  • Meeting transcription
  • Call center analytics
  • Podcast processing

Usage:

How It Works

Speaker diarization segments audio by speaker, answering “who spoke when?” It uses embeddings from sliding windows and clustering to identify speaker turns.

This class supports both:

  • ONNX mode: Load pre-trained models for fast inference
  • Native training mode: Train from scratch using the layer architecture

Constructors

Constructor Summary
SpeakerDiarizer(NeuralNetworkArchitecture<T> architecture, SpeakerDiarizerOptions options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer) Creates a new speaker diarizer in native training mode.
SpeakerDiarizer(NeuralNetworkArchitecture<T> architecture, string modelPath, SpeakerDiarizerOptions options) Creates a new speaker diarizer in native training mode.
SpeakerDiarizer(SpeakerDiarizerOptions options) Creates a new speaker diarizer with legacy options only.

Properties

Property Summary
bool IsOnnxMode { get; } Gets whether the model is operating in ONNX inference mode.
bool SupportsOverlapDetection { get; } Gets whether this model can detect overlapping speech.
double ClusteringThreshold { get; } Gets the clustering threshold.
double MinSegmentDuration { get; } Gets the minimum segment duration in seconds.
double MinTurnDuration { get; } Gets the minimum turn duration in seconds.
int SampleRate { get; } Gets the sample rate.

Methods

Method Summary
DiarizationResult DiarizeLegacy(Tensor<T> audio) Performs diarization on audio (legacy API).
DiarizationResult DiarizeLegacy(Vector<T> audio) Performs diarization on audio (legacy API).
DiarizationResult<T> Diarize(Tensor<T> audio, int? numSpeakers, int minSpeakers, int maxSpeakers) Performs speaker diarization on audio.
DiarizationResult<T> DiarizeWithKnownSpeakers(Tensor<T> audio, IReadOnlyList<SpeakerProfile<T>> knownSpeakers, bool allowUnknownSpeakers) Performs diarization with known speaker profiles.
DiarizationResult<T> RefineDiarization(Tensor<T> audio, DiarizationResult<T> previousResult, T mergeThreshold) Refines diarization result by re-segmenting with different parameters.
IReadOnlyDictionary<string, Tensor<T>> ExtractSpeakerEmbeddings(Tensor<T> audio, DiarizationResult<T> diarizationResult) Gets speaker embeddings for each detected speaker.
ModelMetadata<T> GetModelMetadata() Gets metadata about the model.
Task<DiarizationResult<T>> DiarizeAsync(Tensor<T> audio, int? numSpeakers, int minSpeakers, int maxSpeakers, CancellationToken cancellationToken) Performs speaker diarization asynchronously.
Tensor<T> ForwardForTraining(Tensor<T> input)
void Dispose() Disposes resources.