CTCDecoder<T>
Models & Types · AiDotNet.Audio.SpeechRecognition
CTC (Connectionist Temporal Classification) decoder-based speech recognition model.
Inheritance: NeuralNetworkBase<T> → AudioNeuralNetworkBase<T> → CTCDecoder<T>
Implements: ISpeechRecognizer<T>
For Beginners
Traditional speech recognition required aligning audio with text frame-by-frame, which is expensive and error-prone. CTC removes this requirement:
- The encoder processes audio features (mel-spectrogram) and outputs a probability
distribution over characters for each time frame.
- CTC allows the model to output a special “blank” token when it’s not sure which
character comes next.
- The decoder collapses repeated characters and removes blanks to get the final text.
Example: “h h h - e e - l - l - o” becomes “hello” (- = blank, repeated chars collapsed).
Beam search improves accuracy by considering multiple possible decodings simultaneously.
Usage:
How It Works
CTC (Graves et al., 2006) is the most widely-used alignment-free training criterion for ASR. A CTC decoder pairs a neural encoder (Transformer, Conformer, or LSTM) with a CTC output head and decodes using greedy or beam search with optional language model rescoring. This class provides a standalone CTC-based ASR pipeline with configurable beam width and language model integration.
Constructors
| Constructor | Summary |
|---|---|
CTCDecoder(NeuralNetworkArchitecture<T> architecture, CTCDecoderOptions options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer) |
Creates a CTC decoder model in native training mode. |
CTCDecoder(NeuralNetworkArchitecture<T> architecture, string modelPath, CTCDecoderOptions options) |
Creates a CTC decoder model in native training mode. |
Properties
| Property | Summary |
|---|---|
IReadOnlyList<string> SupportedLanguages { get; } |
|
bool SupportsStreaming { get; } |
|
bool SupportsWordTimestamps { get; } |
Methods
| Method | Summary |
|---|---|
IReadOnlyDictionary<string, T> DetectLanguageProbabilities(Tensor<T> audio) |
|
IStreamingTranscriptionSession<T> StartStreamingSession(string language) |
|
ModelMetadata<T> GetModelMetadata() |
|
Task<TranscriptionResult<T>> TranscribeAsync(Tensor<T> audio, string language, bool? includeTimestamps, CancellationToken cancellationToken) |
|
TranscriptionResult<T> Transcribe(Tensor<T> audio, string language, bool? includeTimestamps) |
|
string DetectLanguage(Tensor<T> audio) |