FunASRNano<T>
Models & Types · AiDotNet.SpeechRecognition.AlibabaASR
Fun-ASR-nano (Tongyi Lab, 2025): an LLM-based recognizer of a SenseVoice-Small audio encoder, a Transformer adaptor and the Qwen3-0.6B decoder, with a CTC decoder whose first-pass hypothesis drives hotword retrieval.
Inheritance: NeuralNetworkBase<T> → AudioNeuralNetworkBase<T> → FunASRNano<T>
Implements: ISpeechRecognizer<T>
For Beginners
Fun-ASR-nano listens to speech, turns it into vectors a language model can read, and lets the language model (Qwen3-0.6B) write the transcript as the answer to a chat prompt that asks for one. A separate, faster CTC decoder produces a rough first transcript that is used to look up the hotwords to put in the prompt.
How It Works
The report (§3, Figure 2) gives four parts: an audio encoder (0.2B parameters for nano), an adaptor of two Transformer
encoder layers, a CTC decoder on the audio encoder, and the LLM decoder. The release (Fun-ASR-Nano-2512’s config and
FunASRNano model) fixes them: SenseVoice-Small’s encoder (50 + 20 SAN-M blocks of width 512 over 80-band
filterbanks stacked seven every six frames); the adaptor, a linear-ReLU-linear projection to the LLM width and two
Transformer encoder layers; Qwen3-0.6B, reading the adaptor’s output in place of the speech inside a ChatML prompt
(“You are a helpful assistant.” as system, the instruction “语音转写:” before the speech); and the CTC decoder, five
Transformer encoder layers and a linear head over a 60,515-token vocabulary on the detached encoder output. The LLM’s
loss is the transcript’s cross-entropy after the prompt. Training follows the report’s §4.2 stages
(TrainingStage): only the modules the stage names are updated, LoRA adapters (rank 16)
stand in for the LLM, and the CTC stage trains the CTC decoder alone with targets in its own vocabulary. Those stages
presuppose a pretrained Qwen3, so the default is joint training of every parameter on the LLM’s loss. Without Qwen’s
151,936-token vocabulary (a smaller VocabSize) the prompt has no text: the LLM reads
the speech alone and its last token ends the answer.
Example
var architecture = new NeuralNetworkArchitecture<float>(
inputType: InputType.OneDimensional,
taskType: NeuralNetworkTaskType.SpeechRecognition,
inputSize: 480000, outputSize: 151936);
var model = new FunASRNano<float>(architecture);
var text = model.Transcribe(waveform).Text;
Constructors
| Constructor | Summary |
|---|---|
FunASRNano(NeuralNetworkArchitecture<T> architecture, FunASRNanoOptions options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer) |
|
FunASRNano(NeuralNetworkArchitecture<T> architecture, string modelPath, FunASRNanoOptions options) |
Properties
| Property | Summary |
|---|---|
IReadOnlyList<string> SupportedLanguages { get; } |
|
bool SupportsStreaming { get; } |
|
bool SupportsWordTimestamps { get; } |
Methods
| Method | Summary |
|---|---|
IReadOnlyDictionary<string, T> DetectLanguageProbabilities(Tensor<T> audio) |
|
IStreamingTranscriptionSession<T> StartStreamingSession(string language) |
|
ModelMetadata<T> GetModelMetadata() |
|
Task<TranscriptionResult<T>> TranscribeAsync(Tensor<T> audio, string language, bool? includeTimestamps, CancellationToken cancellationToken) |
|
Tensor<T> ForwardForTraining(Tensor<T> input) |
|
TranscriptionResult<T> Transcribe(Tensor<T> audio, string language, bool? includeTimestamps) |
Transcribes audio: the LLM’s greedy answer to the transcription prompt, with Hotwords in its context section. |
int[] FirstPassTokens(Tensor<T> audio) |
The CTC decoder’s first-pass hypothesis (ids in the CTC vocabulary), which the report uses to retrieve the hotwords for the prompt. |
string DetectLanguage(Tensor<T> audio) |