RWKVTransducer<T>
Models & Types · AiDotNet.SpeechRecognition.ConformerFamily
RWKV streaming transducer: a linear-attention encoder that is genuinely recurrent, so it streams without chunking or a growing cache.
Inheritance: NeuralNetworkBase<T> → AudioNeuralNetworkBase<T> → RWKVTransducer<T>
Implements: ISpeechRecognizer<T>
For Beginners
To caption speech live, a model must answer as words arrive rather than after the speaker stops. Ordinary attention re-reads the whole recording for each new frame, so it cannot do that cheaply. This keeps a small running summary updated once per frame — the memory it uses does not grow as someone keeps talking, and the live answer matches what you would get from the finished recording.
How It Works
An and Zhang, “Exploring RWKV for Memory Efficient and Low Latency Streaming ASR” (arXiv:2309.14758). The problem they state: “the full-sequence attention mechanism is non-streamable and computationally expensive, thus requiring modifications, such as chunking and caching, for efficient streaming ASR.” RWKV “combines the superior performance of transformers and the inference efficiency of RNNs, which is well-suited for streaming ASR scenarios where the budget for latency and memory is restricted.”
Two properties carry the method and are implemented rather than described.
See RwkvTimeMixing for the recurrence itself.
Constructors
| Constructor | Summary |
|---|---|
RWKVTransducer(NeuralNetworkArchitecture<T> architecture, RWKVTransducerOptions options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer) |
|
RWKVTransducer(NeuralNetworkArchitecture<T> architecture, string modelPath, RWKVTransducerOptions options) |
Properties
| Property | Summary |
|---|---|
IReadOnlyList<string> SupportedLanguages { get; } |
|
bool SupportsStreaming { get; } |
|
bool SupportsWordTimestamps { get; } |
Always false: this model has no word alignment. |
Methods
| Method | Summary |
|---|---|
IReadOnlyDictionary<string, T> DetectLanguageProbabilities(Tensor<T> audio) |
Not supported: this model has no language-identification head. |
IStreamingTranscriptionSession<T> StartStreamingSession(string language) |
|
ModelMetadata<T> GetModelMetadata() |
|
Task<TranscriptionResult<T>> TranscribeAsync(Tensor<T> audio, string language, bool? includeTimestamps, CancellationToken cancellationToken) |
|
TranscriptionResult<T> Transcribe(Tensor<T> audio, string language, bool? includeTimestamps) |
Transcribes audio using RWKVTransducer’s RWKV-enhanced parallel branches. |
string DetectLanguage(Tensor<T> audio) |
Not supported: this model has no language-identification head. |