ParakeetCTC<T>
Models & Types · AiDotNet.SpeechRecognition.NeMo
Parakeet-CTC: NVIDIA NeMo’s Fast Conformer (Rekesh et al. 2023) with a CTC layer.
Inheritance: NeuralNetworkBase<T> → AudioNeuralNetworkBase<T> → ParakeetCTC<T>
Implements: ISpeechRecognizer<T>
For Beginners
Parakeet-CTC turns 16 kHz audio into an 80-band log-mel spectrogram, shortens it 8x with depthwise-separable convolutions, and encodes it with Conformer blocks, each mixing self-attention (long-range context) with a convolution (local patterns). A linear layer gives every frame a probability for each token; collapsing repeats and blanks gives the transcript.
How It Works
Fast Conformer replaces the Conformer front end with 8x depthwise-separable convolution subsampling of 256 channels and narrows the depthwise kernel to 9 (section 3.1). The defaults are parakeet-ctc-0.6b: 24 blocks, width 1024, 8 heads, a 1024-token vocabulary and a CTC layer. SpecAugment masks the features in training (section 3.1). The CTC layer trains with the CTC loss (Graves et al. 2006) and decodes greedily. The paper’s blocks use relative
positional encoding; these use rotary position encoding, the relative scheme
ConformerBlockLayer provides.
Example
var architecture = new NeuralNetworkArchitecture<float>(
inputType: InputType.OneDimensional,
taskType: NeuralNetworkTaskType.SpeechRecognition,
inputSize: 16000, outputSize: 34);
var model = new ParakeetCTC<float>(architecture);
var text = model.Transcribe(waveform).Text;
Constructors
| Constructor | Summary |
|---|---|
ParakeetCTC(NeuralNetworkArchitecture<T> architecture, ParakeetCTCOptions options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer) |
|
ParakeetCTC(NeuralNetworkArchitecture<T> architecture, string modelPath, ParakeetCTCOptions options) |
Properties
| Property | Summary |
|---|---|
IReadOnlyList<string> SupportedLanguages { get; } |
|
bool SupportsStreaming { get; } |
|
bool SupportsWordTimestamps { get; } |
Methods
| Method | Summary |
|---|---|
IReadOnlyDictionary<string, T> DetectLanguageProbabilities(Tensor<T> audio) |
|
IStreamingTranscriptionSession<T> StartStreamingSession(string language) |
|
ModelMetadata<T> GetModelMetadata() |
|
Task<TranscriptionResult<T>> TranscribeAsync(Tensor<T> audio, string language, bool? includeTimestamps, CancellationToken cancellationToken) |
|
Tensor<T> ForwardForTraining(Tensor<T> input) |
|
TranscriptionResult<T> Transcribe(Tensor<T> audio, string language, bool? includeTimestamps) |
Transcribes audio: the encoder and the CTC layer with greedy decoding. |
string DetectLanguage(Tensor<T> audio) |