IVoiceActivityDetector<T>
Interfaces · AiDotNet.Interfaces
Defines the contract for Voice Activity Detection (VAD) models.
Known implementations: MarbleNet<T>, QuailVad<T>, SileroVad<T>, VoiceActivityDetectorBase<T>, WebRTCVad<T>
For Beginners
VAD answers the question “Is someone speaking right now?”
Why VAD is important:
- Speech Recognition: Only process audio when speech is present (saves compute)
- Voice Assistants: Detect when user starts/stops talking
- VoIP/Video Calls: Only transmit audio when speaking (saves bandwidth)
- Transcription: Find speech segments in long recordings
- Speaker Diarization: First step to identify who spoke when
How it works:
- Traditional: Look at energy levels, zero-crossing rate, spectral features
- Modern (Neural): Train a model to classify frames as speech/non-speech
Key metrics:
- Accuracy: How often it’s correct
- False Positive Rate: Saying “speech” when it’s noise (annoying in voice assistants)
- False Negative Rate: Missing actual speech (drops words in transcription)
- Latency: How quickly it detects speech onset
How It Works
Voice Activity Detection determines when speech is present in an audio signal. This is a fundamental building block for many speech processing systems.
Properties
| Property | Summary |
|---|---|
double Threshold { get; set; } |
Gets or sets the detection threshold (0.0 to 1.0). |
int FrameSize { get; } |
Gets the frame size in samples used for detection. |
int MinSilenceDurationMs { get; set; } |
Gets or sets the minimum silence duration in milliseconds. |
int MinSpeechDurationMs { get; set; } |
Gets or sets the minimum speech duration in milliseconds. |
int SampleRate { get; } |
Gets the sample rate this VAD operates at. |
Methods
| Method | Summary |
|---|---|
IReadOnlyList<ValueTuple<int, int>> DetectSpeechSegments(Tensor<T> audio) |
Detects speech segments in a longer audio recording. |
T GetSpeechProbability(Tensor<T> audioFrame) |
Gets the speech probability for an audio frame. |
T[] GetFrameProbabilities(Tensor<T> audio) |
Gets frame-by-frame speech probabilities for the entire audio. |
ValueTuple<bool, T> ProcessChunk(Tensor<T> audioChunk) |
Processes audio in streaming mode, maintaining state between calls. |
bool DetectSpeech(Tensor<T> audioFrame) |
Detects whether speech is present in an audio frame. |
void ResetState() |
Resets internal state for streaming mode. |