RNNTransducer<T>
Models & Types · AiDotNet.Audio.SpeechRecognition
RNN-Transducer (RNN-T) streaming speech recognition model.
Inheritance: NeuralNetworkBase<T> → AudioNeuralNetworkBase<T> → RNNTransducer<T>
Implements: ISpeechRecognizer<T>
For Beginners
RNN-T is a real-time speech recognizer ideal for live transcription. Unlike batch models (like Whisper) that need the whole audio, RNN-T processes speech as it arrives - perfect for live captioning and voice assistants.
It has three parts:
- Encoder - Listens to the audio (converts sound to features)
- Predictor - Remembers what was already said (like a small language model)
- Joiner - Combines both to decide the next output token
Think of it like a court stenographer who listens (encoder), remembers the context (predictor), and types the next word (joiner) - all in real time.
Usage:
How It Works
RNN-Transducer (Graves, 2012; He et al., 2019) combines an audio encoder with a label prediction network and a joint network to produce a streaming ASR system. Unlike CTC, RNN-T can model output dependencies through its prediction network, achieving strong results on LibriSpeech (WER 2.0% with LM) without an external language model. It is the backbone of on-device ASR in Google’s Pixel phones and NVIDIA Riva.
Constructors
| Constructor | Summary |
|---|---|
RNNTransducer(NeuralNetworkArchitecture<T> architecture, RNNTransducerOptions options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer) |
Creates an RNN-T model in native training mode. |
RNNTransducer(NeuralNetworkArchitecture<T> architecture, string modelPath, RNNTransducerOptions options) |
Creates an RNN-T model in native training mode. |
Properties
| Property | Summary |
|---|---|
IReadOnlyList<string> SupportedLanguages { get; } |
|
bool SupportsStreaming { get; } |
|
bool SupportsWordTimestamps { get; } |
Methods
| Method | Summary |
|---|---|
IReadOnlyDictionary<string, T> DetectLanguageProbabilities(Tensor<T> audio) |
|
IStreamingTranscriptionSession<T> StartStreamingSession(string language) |
|
ModelMetadata<T> GetModelMetadata() |
|
Task<TranscriptionResult<T>> TranscribeAsync(Tensor<T> audio, string language, bool? includeTimestamps, CancellationToken cancellationToken) |
|
Tensor<T> ForwardForTraining(Tensor<T> input) |
|
TranscriptionResult<T> Transcribe(Tensor<T> audio, string language, bool? includeTimestamps) |
|
TranscriptionResult<T> TranscribeWithBeamSearch(Tensor<T> audio, int beamWidth, string language, bool? includeTimestamps) |
Transcribes audio with the transducer beam search of Graves 2012 (Algorithm 1): hypotheses over the (frame, label) lattice merged across alignments, keeping beamWidth per frame and returning the one with the highest log-probability per la… |
string DetectLanguage(Tensor<T> audio) |