MarbleNet<T>

Models & Types · AiDotNet.Audio.VoiceActivity

MarbleNet lightweight 1D separable convolutional VAD model (NVIDIA NeMo).

Inheritance: NeuralNetworkBase<T> → AudioNeuralNetworkBase<T> → MarbleNet<T>

Implements: IVoiceActivityDetector<T>

For Beginners

MarbleNet is NVIDIA’s efficient voice activity detector. It uses a special neural network layer (separable convolutions) that makes it very fast while still being accurate. Think of it as a “speech or not?” classifier that can run in real-time even on a phone or small device.

Usage:

How It Works

MarbleNet (Jia et al., 2021, NVIDIA NeMo) is a lightweight 1D time-channel separable convolutional model for voice activity detection. It uses depth-wise separable convolutions with sub-word modeling to achieve state-of-the-art accuracy while being fast enough for real-time streaming on edge devices.

Constructors

Constructor Summary
MarbleNet(NeuralNetworkArchitecture<T> architecture, MarbleNetOptions options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer) Creates a MarbleNet model in native training mode.
MarbleNet(NeuralNetworkArchitecture<T> architecture, string modelPath, MarbleNetOptions options) Creates a MarbleNet model in native training mode.

Properties

Property Summary
double Threshold { get; set; }
int FrameSize { get; }
int MinSilenceDurationMs { get; set; }
int MinSpeechDurationMs { get; set; }

Methods

Method Summary
IReadOnlyList<ValueTuple<int, int>> DetectSpeechSegments(Tensor<T> audio)
ModelMetadata<T> GetModelMetadata()
T GetSpeechProbability(Tensor<T> audioFrame)
T[] GetFrameProbabilities(Tensor<T> audio)
ValueTuple<bool, T> ProcessChunk(Tensor<T> audioChunk)
bool DetectSpeech(Tensor<T> audioFrame)