RobustConformerOptions
Options & Configuration · AiDotNet.SpeechRecognition.Robust
Options for Wang et al. 2021’s noise-robust wav2vec 2.0 (RobustConformer).
Inheritance: ModelOptions → RobustConformerOptions
For Beginners
These options set the size of the speech encoder, the reconstruction module used only in noisy pre-training, and the regularization used while fine-tuning on transcripts.
How It Works
The defaults are section 3.2’s LibriSpeech model: wav2vec 2.0 BASE (12 Transformer blocks of width 768) as
released in wav2vec_small.pt, the reconstruction module of section 2.2 weighted by λ2 = 0.1 (Equation 4), and
section 3.3’s CTC fine-tuning over characters.
Constructors
| Constructor | Summary |
|---|---|
RobustConformerOptions() |
Initializes a new instance with default values. |
RobustConformerOptions(RobustConformerOptions other) |
Initializes a new instance by copying from another instance. |
Properties
| Property | Summary |
|---|---|
OnnxModelOptions OnnxOptions { get; set; } |
ONNX runtime options for the ONNX constructor. |
bool FreezeFeatureEncoder { get; set; } |
Whether fine-tuning keeps the convolutional feature encoder frozen. |
double ActivationDropout { get; set; } |
Dropout inside each feed-forward block. |
double DiversityWeight { get; set; } |
λ1, the diversity loss’s weight in Equation 4 (0.1). |
double DropoutRate { get; set; } |
Dropout on the Transformer’s hidden states and attention weights. |
double FeatureProjectionDropout { get; set; } |
Dropout after the feature projection. |
double FinalDropout { get; set; } |
Dropout before the CTC head. |
double LayerDrop { get; set; } |
Probability of skipping each Transformer block during training. |
double MaskTimeProbability { get; set; } |
Time-mask probability while fine-tuning. |
double ReconstructionWeight { get; set; } |
λ2, the reconstruction loss’s weight in Equation 4 (0.1). |
int EncoderDim { get; set; } |
Transformer width (section 3.2: 768). |
int FeedForwardDim { get; set; } |
Feed-forward width of each block (BASE: 3,072). |
int MaxAudioLengthSeconds { get; set; } |
Longest audio, in seconds, a transcription reads; longer input is truncated. |
int MaxTextLength { get; set; } |
Most tokens a transcription emits. |
int NumAttentionHeads { get; set; } |
Attention heads per block: 12, as in the released wav2vec_small.pt the paper continues from (section 3.2’s text says 8, which does not divide that checkpoint’s attention). |
int NumEncoderLayers { get; set; } |
Transformer blocks (section 3.2: 12). |
int PositionalConvGroups { get; set; } |
Groups of the convolutional positional embedding (16). |
int ReconstructionHidden { get; set; } |
LSTM units per direction of the reconstruction module’s BLSTM layers; the paper does not state them, so the default makes the BLSTM output the encoder width (384 per direction). |
int SampleRate { get; set; } |
Audio sample rate in Hz. |
int VocabSize { get; set; } |
CTC vocabulary size, blank (token 0) included; tokens above 0 decode as Unicode code points. |
string Language { get; set; } |
Language reported for a transcription when none is given. |
string ModelPath { get; set; } |
Path of an exported ONNX model, for the ONNX constructor. |