RevAI<T>
Models & Types · AiDotNet.SpeechRecognition.ProprietaryAPI
Rev AI’s Reverb ASR (Rev 2024): the open-source engine behind Rev’s API, a WeNet joint CTC/attention Conformer with a bidirectional attention decoder.
Inheritance: NeuralNetworkBase<T> → AudioNeuralNetworkBase<T> → RevAI<T>
Implements: ISpeechRecognizer<T>
For Beginners
Reverb turns 16 kHz audio into an 80-band log-mel spectrogram, shortens it 4x with two convolutions, and encodes it with Conformer blocks, each mixing self-attention (long-range context) with a convolution (local patterns). A CTC layer reads the transcript off the frames, and two small decoders, one reading left to right and one right to left, learn to write it too, which makes the encoder better and lets the decoders rescore its guesses.
How It Works
Section 2.2 of the Reverb report: an 18-layer Conformer encoder and a bidirectional attention decoder of 6 Transformer
layers, 3 in each direction, about 280M parameters, trained with WeNet’s joint CTC/attention objective (U2++). The
defaults use width 640 (8 heads, feed-forward 2,560), the width that totals about 280M with those layers, and WeNet’s
defaults for the rest: a kernel-15 convolution, CTC weight 0.3, reverse weight 0.3 and label smoothing 0.1. The
loss is w L_CTC + (1 - w)((1 - r) L_left-to-right + r L_right-to-left). Transcription decodes the left-to-right decoder
greedily; the release also offers CTC prefix beam search with attention rescoring. The release’s language-specific
layers, which control verbatimicity, are not modelled. WeNet’s blocks use relative positional encoding; these use
rotary position encoding, the relative scheme ConformerBlockLayer
provides.
Example
var architecture = new NeuralNetworkArchitecture<float>(
inputType: InputType.OneDimensional,
taskType: NeuralNetworkTaskType.SpeechRecognition,
inputSize: 16000, outputSize: 34);
var model = new RevAI<float>(architecture);
var text = model.Transcribe(waveform).Text;
Constructors
| Constructor | Summary |
|---|---|
RevAI(NeuralNetworkArchitecture<T> architecture, RevAIOptions options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer) |
|
RevAI(NeuralNetworkArchitecture<T> architecture, string modelPath, RevAIOptions options) |
Properties
| Property | Summary |
|---|---|
IReadOnlyList<string> SupportedLanguages { get; } |
|
bool SupportsStreaming { get; } |
|
bool SupportsWordTimestamps { get; } |
Methods
| Method | Summary |
|---|---|
IReadOnlyDictionary<string, T> DetectLanguageProbabilities(Tensor<T> audio) |
|
IStreamingTranscriptionSession<T> StartStreamingSession(string language) |
|
ModelMetadata<T> GetModelMetadata() |
|
Task<TranscriptionResult<T>> TranscribeAsync(Tensor<T> audio, string language, bool? includeTimestamps, CancellationToken cancellationToken) |
|
Tensor<T> ForwardForTraining(Tensor<T> input) |
|
TranscriptionResult<T> Transcribe(Tensor<T> audio, string language, bool? includeTimestamps) |
Transcribes audio: the encoder and the attention decoder with greedy decoding. |
string DetectLanguage(Tensor<T> audio) |