NeMoCitrinet<T>

Models & Types · AiDotNet.SpeechRecognition.NeMo

NeMo Citrinet: 1D time-channel separable convolution CTC model with squeeze-and-excitation.

Inheritance: NeuralNetworkBase<T> → AudioNeuralNetworkBase<T> → NeMoCitrinet<T>

Implements: ISpeechRecognizer<T>

For Beginners

Citrinet extends QuartzNet/Jasper with: (1) 1D time-channel separable convolutions that factorize standard convolutions into depthwise temporal and pointwise channel components; (2) Squeeze-and-Excitation (SE) blocks for channel attention; (3) sub…

How It Works

References:

  • Paper: “Citrinet: Closing the Gap between Non-Autoregressive and Autoregressive End-to-End Models for Automatic Speech Recognition” (Majumdar et al., NVIDIA, 2021)

Citrinet extends QuartzNet/Jasper with: (1) 1D time-channel separable convolutions that factorize standard convolutions into depthwise temporal and pointwise channel components; (2) Squeeze-and-Excitation (SE) blocks for channel attention; (3) sub-word tokenization instead of character-level. The architecture uses 5 blocks of repeated B-sub-blocks, each containing 1D depthwise conv + pointwise conv + SE + residual. CTC decoding on sub-word tokens achieves competitive WER with much faster inference than attention-based models.

Example

// Create a Citrinet model with 1D separable convolutions for CTC ASR
var architecture = new NeuralNetworkArchitecture<double>(
    inputType: InputType.OneDimensional,
    taskType: NeuralNetworkTaskType.SpeechRecognition,
    inputHeight: 16000, inputWidth: 1, inputDepth: 1, outputSize: 1024);
var model = new NeMoCitrinet<double>(architecture);
            
// Or load a pre-trained ONNX model for fast CTC inference
var onnxModel = new NeMoCitrinet<double>(architecture, "citrinet.onnx");

Constructors

Constructor Summary
NeMoCitrinet(NeuralNetworkArchitecture<T> architecture, NeMoCitrinetOptions options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer)
NeMoCitrinet(NeuralNetworkArchitecture<T> architecture, string modelPath, NeMoCitrinetOptions options)

Properties

Property Summary
IReadOnlyList<string> SupportedLanguages { get; }
bool SupportsStreaming { get; }
bool SupportsWordTimestamps { get; }

Methods

Method Summary
IReadOnlyDictionary<string, T> DetectLanguageProbabilities(Tensor<T> audio)
IStreamingTranscriptionSession<T> StartStreamingSession(string language)
ModelMetadata<T> GetModelMetadata()
Task<TranscriptionResult<T>> TranscribeAsync(Tensor<T> audio, string language, bool? includeTimestamps, CancellationToken cancellationToken)
TranscriptionResult<T> Transcribe(Tensor<T> audio, string language, bool? includeTimestamps) Transcribes audio using 1D time-channel separable convolutions with CTC decoding.
string DetectLanguage(Tensor<T> audio)