ASTOptions
Options & Configuration · AiDotNet.Audio.Classification
Configuration options for the AST (Audio Spectrogram Transformer) models, AST and
ASTModel, which run one network.
Inheritance: ModelOptions → ASTOptions
For Beginners
AST treats an audio spectrogram as an image. It cuts the spectrogram into overlapping patches, then uses a Transformer to relate them, and scores every sound label independently.
Example usage:
How It Works
AST (Gong, Chung and Glass, Interspeech 2021) applies a Vision Transformer to audio. The defaults are the paper’s AudioSet configuration: 128-bin Kaldi filterbanks of 10-second clips (1024 frames), 16×16 patches with an overlap of 6, the 768-wide 12-layer DeiT body with the class and distillation tokens averaged, inputs normalized to mean 0 and standard deviation 0.5, and training with mixup and SpecAugment.
References:
- Paper: “AST: Audio Spectrogram Transformer” (Gong et al., Interspeech 2021)
- Repository:
https://github.com/YuanGongND/ast
Constructors
| Constructor | Summary |
|---|---|
ASTOptions() |
Creates an instance with the default settings. |
ASTOptions(ASTOptions other) |
Copies the settings from another options instance. |
Properties
| Property | Summary |
|---|---|
OnnxModelOptions OnnxOptions { get; set; } |
Gets or sets ONNX runtime options. |
WindowType Window { get; set; } |
Gets or sets the filterbank’s analysis window. |
double DatasetMean { get; set; } |
Gets or sets the dataset filterbank mean the input is normalized by (AudioSet’s, from the official recipe). |
double DatasetStd { get; set; } |
Gets or sets the dataset filterbank standard deviation (AudioSet’s). |
double MaxGradNorm { get; set; } |
Gets or sets the training gradient norm limit. |
double MixupRate { get; set; } |
Gets or sets the probability that a training clip is mixed with another (0.5, §3.1.1). |
double Threshold { get; set; } |
Gets or sets the confidence threshold for event detection. |
double WindowOverlap { get; set; } |
Gets or sets the window overlap ratio (0-1) for event detection. |
double WindowSize { get; set; } |
Gets or sets the window size in seconds for event detection. |
int EmbeddingDim { get; set; } |
Gets or sets the Transformer width (768). |
int FeedForwardDim { get; set; } |
Gets or sets the MLP width of each block (3072, four times the width). |
int FrequencyMaskLength { get; set; } |
Gets or sets the maximum frequency-mask length in mel bins (48, §3.1.1). |
int FrequencyStride { get; set; } |
Gets or sets the patch stride along frequency (10: an overlap of 6, §2.1). |
int NumAttentionHeads { get; set; } |
Gets or sets the number of attention heads (12). |
int NumEncoderLayers { get; set; } |
Gets or sets the number of Transformer blocks (12). |
int NumMels { get; set; } |
Gets or sets the number of mel bins of the Kaldi filterbank (128, §2.1). |
int PatchSize { get; set; } |
Gets or sets the square patch size (16). |
int SampleRate { get; set; } |
Gets or sets the audio sample rate in Hz (16 kHz in the paper). |
int TargetLength { get; set; } |
Gets or sets the number of filterbank frames the network takes (1024: 10 s at the 10 ms frame shift). |
int TimeMaskLength { get; set; } |
Gets or sets the maximum time-mask length in frames (192, §3.1.1). |
int TimeStride { get; set; } |
Gets or sets the patch stride along time (10). |
string ModelPath { get; set; } |
Gets or sets the path to a pre-trained ONNX model file. |
string[] CustomLabels { get; set; } |
Gets or sets custom event labels. |