CodeSwitchingASROptions
Options & Configuration · AiDotNet.SpeechRecognition.Specialized
Options for code-switching ASR, following Zeng et al., “On the End-to-End Solution to Mandarin-English Code-switching Speech Recognition” (Interspeech 2019, arXiv:1811.00241).
Inheritance: ModelOptions → CodeSwitchingASROptions
For Beginners
Code-switching is when a speaker mixes languages inside one sentence. These settings control three things the model does about that: how it balances two different ways of reading speech (CTC and attention), how much effort it spends explicitly working out WHICH language it is hearing at each moment, and how the two languages’ writing systems share the output alphabet.
How It Works
Defaults follow the paper: a hybrid CTC/attention objective with a CTC weight of 0.2, an auxiliary language-identification task weighted at 0.2, and a 3k BPE inventory for English alongside Mandarin characters.
Constructors
| Constructor | Summary |
|---|---|
CodeSwitchingASROptions() |
Initializes a new instance with default values. |
CodeSwitchingASROptions(CodeSwitchingASROptions other) |
Initializes a new instance by copying from another instance. |
Properties
| Property | Summary |
|---|---|
OnnxModelOptions OnnxOptions { get; set; } |
Gets or sets the ONNX Runtime settings used when the model is loaded from an ONNX file. |
bool SharedLidAttention { get; set; } |
Gets or sets whether the LID head shares the ASR decoder’s attention (the paper’s LID_shared) rather than using its own (LID_indep). |
double CtcWeight { get; set; } |
Gets or sets the CTC branch’s weight in the joint objective L = (1 - CtcWeight) * L_att + CtcWeight * L_ctc + LidWeight * L_lid. |
double DropoutRate { get; set; } |
Gets or sets the dropout on encoder layer outputs and decoder embeddings during training (ESPnet’s dropout-rate). |
double LidWeight { get; set; } |
Gets or sets lambda_2, the weight of the auxiliary language-identification loss. |
int AttentionConvChannels { get; set; } |
Gets or sets the channels of the attention’s location convolution (ESPnet’s aconv_chans). |
int AttentionConvFilters { get; set; } |
Gets or sets the half-width of the location convolution’s kernel (ESPnet’s aconv_filts). |
int AttentionDim { get; set; } |
Gets or sets the width of the location-aware attention (ESPnet’s adim). |
int DecoderDim { get; set; } |
Gets or sets the LSTM units of the one-layer attention decoder (and of the LID decoder). |
int EncoderDim { get; set; } |
Gets or sets the LSTM units per direction of each encoder BLSTM layer. |
int EncoderProjection { get; set; } |
Gets or sets the width the encoder output is projected to (ESPnet’s eprojs). |
int EnglishBpeVocabSize { get; set; } |
Gets or sets the number of English BPE subword units. |
int MandarinCharVocabSize { get; set; } |
Gets or sets the number of Mandarin character units. |
int MandarinTokenCount { get; } |
The number of Mandarin token ids, i.e. |
int MaxAudioLengthSeconds { get; set; } |
Gets or sets the longest clip, in seconds, of the reference configuration. |
int MaxTextLength { get; set; } |
Gets or sets the most tokens decoded for one utterance. |
int NumDecoderLayers { get; set; } |
Gets or sets the stacked LSTM layers of the attention decoder. |
int NumEncoderLayers { get; set; } |
Gets or sets the number of encoder BLSTM layers. |
int NumMels { get; set; } |
Gets or sets the log-mel bins of the input features. |
int SampleRate { get; set; } |
Gets or sets the sample rate, in Hz, that input audio is expected at. |
int VocabSize { get; set; } |
Gets or sets the total output inventory. |
string Language { get; set; } |
Gets or sets the language code used for speech processing. |
string ModelPath { get; set; } |
Gets or sets the path of the pretrained ONNX model. |
string[] Vocabulary { get; set; } |
Gets or sets the text of each token id (Mandarin characters, then English BPE units); ids past the list decode as Unicode code points. |