Canary<T>
Models & Types · AiDotNet.Audio.SpeechRecognition
Canary multilingual speech recognition and translation model from NVIDIA.
Inheritance: NeuralNetworkBase<T> → AudioNeuralNetworkBase<T> → Canary<T>
Implements: ISpeechRecognizer<T>
For Beginners
Canary is like a multilingual transcription assistant. It can listen to speech in many languages and either transcribe it (write down what was said) or translate it into another language, all with a single model. You tell it what you want via a prompt.
Usage:
How It Works
Canary (NVIDIA, 2024) is a multilingual ASR/ST model based on the Fast Conformer encoder with a multi-task decoder. It supports transcription and translation across many languages using a single unified architecture with task-specific prompting, achieving strong WER scores across English, German, Spanish, and French.
Constructors
| Constructor | Summary |
|---|---|
Canary(NeuralNetworkArchitecture<T> architecture, CanaryOptions options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer) |
Creates a Canary model in native training mode. |
Canary(NeuralNetworkArchitecture<T> architecture, string modelPath, CanaryOptions options) |
Creates a Canary model in native training mode. |
Properties
| Property | Summary |
|---|---|
IReadOnlyList<string> SupportedLanguages { get; } |
|
bool IsOnnxMode { get; } |
|
bool SupportsStreaming { get; } |
|
bool SupportsWordTimestamps { get; } |
|
int SampleRate { get; } |
Methods
| Method | Summary |
|---|---|
IReadOnlyDictionary<string, T> DetectLanguageProbabilities(Tensor<T> audio) |
|
IStreamingTranscriptionSession<T> StartStreamingSession(string language) |
|
ModelMetadata<T> GetModelMetadata() |
|
Task<TranscriptionResult<T>> TranscribeAsync(Tensor<T> audio, string language, bool? includeTimestamps, CancellationToken cancellationToken) |
|
TranscriptionResult<T> Transcribe(Tensor<T> audio, string language, bool? includeTimestamps) |
|
string DetectLanguage(Tensor<T> audio) |