Speech Recognition

181 public types in this namespace, organized by kind.

Models & Types (92)

Type Summary
AVHuBERT<T> AV-HuBERT: a Transformer over fused audio and lip-video features, pre-trained by masked multimodal cluster prediction, with a CTC projection for recognition from audio, lips or both.
AWSTranscribe<T> An Amazon Transcribe stand-in: a streaming Conformer transducer, since Amazon has not disclosed Transcribe’s architecture.
AssemblyAIUniversal2<T> AssemblyAI Universal-2: a 600M-parameter Conformer RNN-T, the configuration of Universal-1 (Ramirez et al.
AudioPaLM<T> AudioPaLM’s speech recognition (Rubenstein et al.
AzureSpeechSTT<T> Azure Speech’s nearest disclosed recognizer: Microsoft’s streaming Conformer transducer (Chen et al.
BESTRQ<T> BEST-RQ: a Conformer speech encoder pre-trained to predict the labels a fixed random-projection quantizer assigns to the masked input, with a CTC layer for recognition.
Branchformer<T> Branchformer (Peng et al.
CIFDecoder<T> The CIF-based recognizer of Dong and Xu (2020): continuous integrate-and-fire between a self-attention encoder and a non-autoregressive decoder.
CIFEncoder<T> CIF (Continuous Integrate-and-Fire) encoder for soft monotonic alignment in ASR.
CTCSegmentation<T> CTC-segmentation (Kurzinger et al.
CanaryFlash<T> Canary-1B-Flash: NVIDIA’s Canary (Puvvada et al.
CanaryQwen<T> Canary-Qwen-2.5B (NVIDIA, 2025): a speech-augmented language model (SALM, Chen et al.
Chirp2<T> Chirp 2: Google Cloud’s second-generation Chirp recognizer, a USM-family Conformer with a CTC layer.
Chirp3<T> Chirp 3: Google Cloud’s third-generation Chirp recognizer, a USM-family Conformer with a CTC layer.
Chirp<T> Chirp: Google Cloud’s first-generation Chirp recognizer, the 2B-parameter Universal Speech Model (USM) with a CTC layer, served through Speech-to-Text V2.
CodeSwitchingASR<T> Code-switching ASR: a hybrid CTC/attention model trained jointly with an EXPLICIT language-identification task, for speech that mixes languages inside one utterance.
ConformerCTC<T> Conformer-CTC: the Conformer encoder (Gulati et al.
ConformerTransducer<T> Conformer-Transducer (Gulati et al.
ContextNet<T> ContextNet: CNN encoder with squeeze-and-excitation and global context for ASR.
ConvTransformer<T> Transformers with convolutional context (Mohamed et al.
Data2VecASR<T> Data2Vec ASR: general-purpose self-supervised learning
DeepgramNova2<T> A Deepgram Nova-2 stand-in: a Transformer encoder with a CTC layer, since Deepgram has disclosed only that Nova-2 is a Transformer-based end-to-end model.
DistilWhisper<T> Distil-Whisper (Gandhi et al.
EBranchformer<T> E-Branchformer (Kim et al.
ESPnetASR<T> ESPnet ASR (Watanabe et al.
EfficientConformer<T> Efficient Conformer with progressive temporal downsampling and grouped attention.
EmformerRNNT<T> Emformer RNN-T (Shi et al.
FastEmit<T> FastEmit (Yu et al.
FasterWhisper<T> faster-whisper: SYSTRAN’s CTranslate2 inference engine for Whisper checkpoints, here at large-v3.
FireRedASRLLM<T> FireRedASR-LLM (FireRed Team, 2025): the Encoder-Adapter-LLM recognizer of FireRedASR, which feeds the FireRedASR-AED Conformer encoder’s output through an adapter into Qwen2-7B-Instruct.
FireRedASR<T> FireRedASR-AED (Xu et al.
FunASRNano<T> Fun-ASR-nano (Tongyi Lab, 2025): an LLM-based recognizer of a SenseVoice-Small audio encoder, a Transformer adaptor and the Qwen3-0.6B decoder, with a CTC decoder whose first-pass hypothesis drives hotword retrieval.
GladiaASR<T> Gladia’s speech-to-text: a hosted recognizer built on Whisper large-v3.
GoogleSpeechV2<T> Google Cloud Speech-to-Text V2: Google’s hosted recognizer, served on its Chirp models, which are the Universal Speech Model (USM) with a CTC layer.
GraniteSpeech<T> Granite-speech (Saon et al., IBM 2025): a conformer CTC acoustic encoder, a window Q-Former speech modality adapter and a Granite LLM with LoRA adapters, which writes the transcript of the audio it is given in its chat prompt.
GroqWhisper<T> Groq’s hosted Whisper: openai-whisper large-v3 served on Groq’s inference hardware.
HuBERTASR<T> HuBERT ASR: Hidden-Unit BERT for self-supervised speech representation
InterCTC<T> InterCTC (Lee and Watanabe 2021): a CTC encoder regularized by a second CTC loss on its middle layer.
KeywordSpotting<T> Keyword Spotting: lightweight wake-word and command detection
KotobaWhisper<T> Kotoba-Whisper: Distil-Whisper’s recipe applied to Japanese, a large-v3 encoder with a 2-layer decoder.
MMS<T> MMS: Massively Multilingual Speech covering 1100+ languages
MedicalASR<T> Medical-conversation ASR: Chiu et al.
MoonshineBase<T> Moonshine Base (Jeffries et al.
Moonshine<T> Moonshine Tiny (Jeffries et al.
Moshi<T> Moshi’s streaming speech recognizer (Défossez et al.
NeMoCitrinet<T> NeMo Citrinet: 1D time-channel separable convolution CTC model with squeeze-and-excitation.
NeMoMultitask<T> NeMo’s multitask attention encoder-decoder, Canary-1B (Puvvada et al.
NemotronSpeech<T> Nemotron Speech Streaming: NVIDIA’s cache-aware Fast Conformer (Noroozi et al.
NoiseRobustASR<T> Noise-robust speech recognition as Whisper (Radford et al.
OLMoASR<T> OLMoASR (Ngo et al., Ai2 2025): Whisper’s encoder-decoder, with Whisper’s English-only tokenizer, trained from scratch on the curated open OLMoASR-Mix.
OWSM<T> OWSM v3.1 (Peng et al.
OmnilangualASR<T> Omnilingual ASR’s CTC recognizer: a wav2vec 2.0 encoder over the raw waveform with a single linear CTC layer.
ParaformerLarge<T> Paraformer-large (Gao et al.
Paraformer<T> Paraformer (Gao et al.
ParakeetCTC<T> Parakeet-CTC: NVIDIA NeMo’s Fast Conformer (Rekesh et al.
ParakeetRNNT<T> Parakeet-RNNT: NVIDIA NeMo’s Fast Conformer (Rekesh et al.
ParakeetTDT<T> Parakeet-TDT: NVIDIA NeMo’s Fast Conformer with a token-and-duration transducer (Xu et al.
Phi4Audio<T> Phi-4-multimodal’s speech recognition (Abdin et al., Microsoft 2025): a conformer audio encoder and a projector that feed Phi-4-Mini, which carries the speech LoRA adapters (“Mixture of LoRAs”) and writes the transcript.
Qwen3ASRSmall<T> Qwen3-ASR-0.6B (Qwen Team, 2026): an LLM-based recognizer of the AuT audio encoder and the Qwen3-0.6B decoder, which names the language and then writes the transcript.
Qwen3ASR<T> Qwen3-ASR-1.7B (Qwen Team, 2026): an LLM-based recognizer of the AuT audio encoder and the Qwen3-1.7B decoder, which names the language and then writes the transcript.
RWKVTransducer<T> RWKV streaming transducer: a linear-attention encoder that is genuinely recurrent, so it streams without chunking or a growing cache.
RevAI<T> Rev AI’s Reverb ASR (Rev 2024): the open-source engine behind Rev’s API, a WeNet joint CTC/attention Conformer with a bidirectional attention decoder.
RobustConformer<T> Noise-robust wav2vec 2.0 (Wang et al.
SALM<T> SALM (Chen et al., 2023): a speech-augmented language model that feeds a Fast Conformer encoder’s output through a Conformer modality adapter into a Megatron GPT with LoRA, for multitask speech recognition and translation with in-context le…
SPIRAL<T> SPIRAL: a convolution-Transformer speech encoder pre-trained to make its representation of perturbed speech match a moving-average teacher’s representation of the clean speech, with a convolutional CTC classifier.
SambaASR<T> Samba-ASR: Mamba-based state-space model for speech recognition
SarvamASR<T> A Sarvam speech-to-text stand-in: a Conformer encoder with a CTC layer over Unicode characters of the Indic scripts, since Sarvam has not disclosed Saarika’s architecture.
SeACo<T> SeACo-Paraformer (Shi et al.
SeedASR<T> Seed-ASR (Bai et al., 2024): ByteDance’s audio-conditioned LLM recognizer, which feeds the LUISE Conformer encoder’s output through a frame-splicing converter into a large language model.
SelfConditionedCTC<T> Self-conditioned CTC (Nozaki and Komatsu 2021): intermediate CTC predictions fed back into the encoder.
SenseVoiceLarge<T> SenseVoice-Large (FunAudioLLM, 2024): an autoregressive encoder-decoder for multilingual recognition, language identification, speech emotion recognition and audio event detection, its tasks selected by decoder input tokens.
SenseVoice<T> SenseVoice-Small (An et al.
SpeakerDiarizedASR<T> Speaker-Diarized ASR: who-spoke-what transcription
SpeechBrain<T> SpeechBrain ASR (Ravanelli et al.
SpeechGPT<T> SpeechGPT’s speech recognition (Zhang et al.
SpeechmaticsASR<T> A Speechmatics stand-in after Ursa’s disclosed design: an efficient Transformer encoder whose acoustic model gives phoneme probabilities, here a Conformer with a CTC layer over phonemes.
Squeezeformer<T> Squeezeformer speech recognition model with temporal U-Net structure.
StreamingConformer<T> A streaming Conformer transducer: the Conformer of Gulati et al.
StreamingZipformer<T> Streaming Zipformer (Yao et al.
TDTDecoder<T> Token-and-Duration Transducer (Xu et al.
USM<T> Google USM’s CTC recognizer (USM-CTC): a 2B-parameter Conformer pre-trained with BEST-RQ, with a projection and a CTC layer.
UniSpeech<T> UniSpeech: unified pre-training for speech representation learning
VoxtLM<T> VoxtLM: one decoder-only language model for speech recognition, synthesis, and speech and text continuation (Maiti et al.
WavLMASR<T> WavLM ASR: speech pre-training with denoising and attention
WavLMRobust<T> WavLM-Robust: noise-robust speech recognition via denoising pre-training
WhisperCPP<T> whisper.cpp: Georgi Gerganov’s C/C++ port of Whisper, here at the base checkpoint it ships by default.
WhisperLargeV3Turbo<T> OpenAI’s Whisper large-v3-turbo: large-v3’s encoder with the decoder pruned to 4 layers and fine-tuned.
WhisperLargeV3<T> OpenAI’s Whisper large-v3: a 1.55B-parameter encoder-decoder Transformer over 128-bin log-mel windows.
WhisperLive<T> WhisperLive: Collabora’s near-real-time Whisper server, re-transcribing the growing audio buffer as chunks arrive.
WhisperTimestamped<T> whisper-timestamped: Whisper with word timestamps by dynamic time warping over cross-attention.
WhisperX<T> WhisperX (Bain et al.
XLSR<T> XLS-R: cross-lingual speech representation at scale

Options & Configuration (89)

Type Summary
AVHuBERTOptions Options for AV-HuBERT: audio-visual masked multimodal cluster prediction, with a CTC projection.
AWSTranscribeOptions Options for the Amazon Transcribe stand-in, a streaming Conformer transducer (Gulati et al.
AssemblyAIUniversal2Options Options for AssemblyAI Universal-2.
AzureSpeechSTTOptions Options for the nearest disclosed architecture to Azure Speech, Microsoft’s streaming Conformer transducer (Chen et al.
BESTRQOptions Options for BEST-RQ: Conformer XL pre-trained with a random-projection quantizer, with a CTC layer.
BranchformerOptions Options for Branchformer.
CIFDecoderOptions Options for the CIF-based recognizer of Dong and Xu (2020).
CIFEncoderOptions Options for CIF (Continuous Integrate-and-Fire) encoder for soft monotonic alignment (Dong & Xu, 2020).
CTCSegmentationOptions Options for CTC-segmentation.
CanaryFlashOptions Options for Canary-1B-Flash.
CanaryQwenOptions Options for Canary-Qwen: canary-1b-flash’s Fast Conformer encoder, a linear projection and Qwen3-1.7B.
Chirp2Options Options for Chirp 2, built as Google USM-CTC: a BEST-RQ pre-trained Conformer with a CTC layer.
Chirp3Options Options for Chirp 3, built as Google USM-CTC: a BEST-RQ pre-trained Conformer with a CTC layer.
ChirpOptions Options for Chirp, built as Google USM-CTC: a BEST-RQ pre-trained Conformer with a CTC layer.
CodeSwitchingASROptions Options for code-switching ASR, following Zeng et al., “On the End-to-End Solution to Mandarin-English Code-switching Speech Recognition” (Interspeech 2019, arXiv:1811.00241).
ConformerCTCOptions Options for Conformer-CTC.
ConformerTransducerOptions Options for Conformer-Transducer.
ContextNetOptions Options for ContextNet CNN encoder with squeeze-and-excitation (Han et al., 2020).
ConvTransformerOptions Options for ConvTransformer.
Data2VecASROptions Options for data2vec (audio) fine-tuned for speech recognition with CTC.
DeepgramNova2Options Options for the Deepgram Nova-2 stand-in, a Transformer encoder with a CTC layer (Deepgram has disclosed only a Transformer-based end-to-end model).
DistilWhisperOptions Options for Distil-Whisper, built on openai-whisper’s network.
EBranchformerOptions Options for E-Branchformer.
ESPnetASROptions Options for ESPnet ASR.
EfficientConformerOptions Options for Efficient Conformer with progressive downsampling (Burchi & Vielzeuf, 2021).
EmformerRNNTOptions Options for Emformer RNN-T.
FastEmitOptions Options for FastEmit.
FasterWhisperOptions Options for faster-whisper, built on openai-whisper’s network.
FireRedASRLLMOptions Options for FireRedASR-LLM: the FireRedASR-AED encoder, an adapter and the Qwen2-7B-Instruct decoder.
FireRedASROptions Options for FireRedASR-AED.
FunASRNanoOptions Options for Fun-ASR-nano: an LLM-based recognizer of a SenseVoice-Small encoder, an adaptor and Qwen3-0.6B.
GladiaASROptions Options for Gladia, built on openai-whisper’s network.
GoogleSpeechV2Options Options for Google Speech-to-Text V2, built as Google USM-CTC: a BEST-RQ pre-trained Conformer with a CTC layer.
GraniteSpeechOptions Options for Granite-speech: a conformer CTC encoder, a window Q-Former projector and a Granite LLM.
GroqWhisperOptions Options for Groq Whisper, built on openai-whisper’s network.
HuBERTASROptions Options for HuBERT fine-tuned for speech recognition with CTC.
InterCTCOptions Options for InterCTC.
KeywordSpottingOptions Options for Keyword Spotting: lightweight wake-word and command detection.
KotobaWhisperOptions Options for Kotoba-Whisper, built on openai-whisper’s network.
MMSOptions Options for MMS: speech recognition in 1,000+ languages.
MedicalASROptions Options for MedicalASR, Chiu et al.
MoonshineBaseOptions Options for Moonshine Base.
MoonshineOptions Options for Moonshine Tiny.
NeMoCitrinetOptions Options for NeMo Citrinet (NVIDIA, 2021): 1D time-channel separable convolution CTC model.
NeMoMultitaskOptions Options for NeMo multitask AED (Canary-1B).
NemotronSpeechOptions Options for Nemotron Speech Streaming: a cache-aware Fast Conformer with an RNN-T decoder.
NoiseRobustASROptions Options for Noise-robust ASR, built on openai-whisper’s network.
OLMoASROptions Options for OLMoASR: Whisper’s encoder-decoder trained on the open OLMoASR-Mix.
OWSMOptions Options for OWSM v3.1.
OmnilangualASROptions Options for Omnilingual ASR’s CTC recognizer.
ParaformerLargeOptions Options for Paraformer-large: the released 220M-parameter Mandarin Paraformer.
ParaformerOptions Options for Paraformer: fast and accurate parallel Transformer for non-autoregressive ASR.
ParakeetCTCOptions Options for Parakeet-CTC.
ParakeetRNNTOptions Options for Parakeet-RNNT.
ParakeetTDTOptions Options for Parakeet-TDT.
Phi4AudioOptions Options for Phi-4-multimodal’s speech path: a conformer audio encoder, a projector and Phi-4-Mini.
Qwen3ASROptions Options for Qwen3-ASR-1.7B: an AuT audio encoder, a projector and the Qwen3-1.7B decoder (Qwen3-ASR technical report, 2026).
Qwen3ASRSmallOptions Options for Qwen3-ASR-0.6B: an AuT audio encoder, a projector and the Qwen3-0.6B decoder (Qwen3-ASR technical report, 2026).
RWKVTransducerOptions Options for the RWKV streaming transducer, An and Zhang, “Exploring RWKV for Memory Efficient and Low Latency Streaming ASR” (arXiv:2309.14758).
RevAIOptions Options for Rev AI’s Reverb ASR.
RobustConformerOptions Options for Wang et al.
SALMOptions Options for SALM: a Fast Conformer audio encoder, a Conformer modality adapter and a Megatron GPT decoder.
SPIRALOptions Options for SPIRAL: self-supervised perturbation-invariant pre-training of a convolution-Transformer speech encoder, with a convolutional CTC classifier.
SambaASROptions Options for Samba-ASR: Mamba-based state-space model for speech recognition.
SarvamASROptions Options for the Sarvam speech-to-text stand-in, a Conformer encoder with a CTC layer over Indic-script characters.
SeACoOptions Options for SeACo-Paraformer: Paraformer-large with a hotword bias encoder, bias decoder and bias output layer.
SeedASROptions Options for Seed-ASR: the LUISE Conformer encoder, the frame-splicing converter and a decoder-only LLM.
SelfConditionedCTCOptions Options for Self-conditioned CTC.
SenseVoiceLargeOptions Options for SenseVoice-Large: the autoregressive encoder-decoder SenseVoice (FunAudioLLM, 2024).
SenseVoiceOptions Options for SenseVoice: multi-task speech understanding model.
SpeakerDiarizedASROptions Options for Speaker-Diarized ASR: who-spoke-what transcription.
SpeechBrainOptions Options for SpeechBrain ASR.
SpeechmaticsASROptions Options for the Speechmatics stand-in after Ursa’s disclosed design, a Conformer encoder with a CTC layer over phonemes.
SqueezeformerOptions Configuration options for the Squeezeformer speech recognition model.
StreamingConformerOptions Options for Streaming Conformer.
StreamingZipformerOptions Options for the streaming Zipformer transducer.
TDTDecoderOptions Options for TDT.
USMOptions Options for Google USM-CTC: a BEST-RQ pre-trained Conformer with a CTC layer.
UniSpeechOptions Options for UniSpeech fine-tuned for speech recognition with CTC.
VoxtLMOptions Options for VoxtLM: one decoder-only language model over speech units and text.
WavLMASROptions Options for WavLM fine-tuned for speech recognition with CTC.
WavLMRobustOptions Options for WavLM-Robust: WavLM fine-tuned with CTC for noisy, reverberant and overlapped speech.
WhisperCPPOptions Options for whisper.cpp, built on openai-whisper’s network.
WhisperLargeV3Options Options for Whisper large-v3, built on openai-whisper’s network.
WhisperLargeV3TurboOptions Options for Whisper large-v3-turbo, built on openai-whisper’s network.
WhisperLiveOptions Options for WhisperLive, built on openai-whisper’s network.
WhisperTimestampedOptions Options for whisper-timestamped, built on openai-whisper’s network.
WhisperXOptions Options for WhisperX, built on openai-whisper’s network.
XLSROptions Options for XLS-R: cross-lingual speech representation at scale.