MarbleNet<T>
Models & Types · AiDotNet.Audio.VoiceActivity
MarbleNet lightweight 1D separable convolutional VAD model (NVIDIA NeMo).
Inheritance: NeuralNetworkBase<T> → AudioNeuralNetworkBase<T> → MarbleNet<T>
Implements: IVoiceActivityDetector<T>
For Beginners
MarbleNet is NVIDIA’s efficient voice activity detector. It uses a special neural network layer (separable convolutions) that makes it very fast while still being accurate. Think of it as a “speech or not?” classifier that can run in real-time even on a phone or small device.
Usage:
How It Works
MarbleNet (Jia et al., 2021, NVIDIA NeMo) is a lightweight 1D time-channel separable convolutional model for voice activity detection. It uses depth-wise separable convolutions with sub-word modeling to achieve state-of-the-art accuracy while being fast enough for real-time streaming on edge devices.
Constructors
| Constructor | Summary |
|---|---|
MarbleNet(NeuralNetworkArchitecture<T> architecture, MarbleNetOptions options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer) |
Creates a MarbleNet model in native training mode. |
MarbleNet(NeuralNetworkArchitecture<T> architecture, string modelPath, MarbleNetOptions options) |
Creates a MarbleNet model in native training mode. |
Properties
| Property | Summary |
|---|---|
double Threshold { get; set; } |
|
int FrameSize { get; } |
|
int MinSilenceDurationMs { get; set; } |
|
int MinSpeechDurationMs { get; set; } |
Methods
| Method | Summary |
|---|---|
IReadOnlyList<ValueTuple<int, int>> DetectSpeechSegments(Tensor<T> audio) |
|
ModelMetadata<T> GetModelMetadata() |
|
T GetSpeechProbability(Tensor<T> audioFrame) |
|
T[] GetFrameProbabilities(Tensor<T> audio) |
|
ValueTuple<bool, T> ProcessChunk(Tensor<T> audioChunk) |
|
bool DetectSpeech(Tensor<T> audioFrame) |