CTCDecoder<T>

Models & Types · AiDotNet.Audio.SpeechRecognition

CTC (Connectionist Temporal Classification) decoder-based speech recognition model.

Inheritance: NeuralNetworkBase<T> → AudioNeuralNetworkBase<T> → CTCDecoder<T>

Implements: ISpeechRecognizer<T>

For Beginners

Traditional speech recognition required aligning audio with text frame-by-frame, which is expensive and error-prone. CTC removes this requirement:

  1. The encoder processes audio features (mel-spectrogram) and outputs a probability

distribution over characters for each time frame.

  1. CTC allows the model to output a special “blank” token when it’s not sure which

character comes next.

  1. The decoder collapses repeated characters and removes blanks to get the final text.

Example: “h h h - e e - l - l - o” becomes “hello” (- = blank, repeated chars collapsed).

Beam search improves accuracy by considering multiple possible decodings simultaneously.

Usage:

How It Works

CTC (Graves et al., 2006) is the most widely-used alignment-free training criterion for ASR. A CTC decoder pairs a neural encoder (Transformer, Conformer, or LSTM) with a CTC output head and decodes using greedy or beam search with optional language model rescoring. This class provides a standalone CTC-based ASR pipeline with configurable beam width and language model integration.

Constructors

Constructor Summary
CTCDecoder(NeuralNetworkArchitecture<T> architecture, CTCDecoderOptions options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer) Creates a CTC decoder model in native training mode.
CTCDecoder(NeuralNetworkArchitecture<T> architecture, string modelPath, CTCDecoderOptions options) Creates a CTC decoder model in native training mode.

Properties

Property Summary
IReadOnlyList<string> SupportedLanguages { get; }
bool SupportsStreaming { get; }
bool SupportsWordTimestamps { get; }

Methods

Method Summary
IReadOnlyDictionary<string, T> DetectLanguageProbabilities(Tensor<T> audio)
IStreamingTranscriptionSession<T> StartStreamingSession(string language)
ModelMetadata<T> GetModelMetadata()
Task<TranscriptionResult<T>> TranscribeAsync(Tensor<T> audio, string language, bool? includeTimestamps, CancellationToken cancellationToken)
TranscriptionResult<T> Transcribe(Tensor<T> audio, string language, bool? includeTimestamps)
string DetectLanguage(Tensor<T> audio)