BEATsOptions
Options & Configuration · AiDotNet.Audio.Classification
Configuration options for the BEATs (Audio Pre-Training with Acoustic Tokenizers) model.
Inheritance: ModelOptions → BEATsOptions
For Beginners
BEATs is a powerful model for recognizing sounds in audio. It works by:
- Breaking audio into small patches (like cutting a spectrogram into tiles)
- Using a Transformer (the same architecture behind ChatGPT) to understand relationships between patches
- Learning what sounds are present by comparing against known audio patterns
You can use BEATs in two ways:
- ONNX mode: Load a pre-trained model for instant inference
- Native mode: Train from scratch on your own audio data
Example usage:
How It Works
BEATs (Chen et al., ICML 2023) is a state-of-the-art audio classification model that uses iterative self-distillation between an acoustic tokenizer and an audio SSL model. It achieves 50.6% mAP on AudioSet-2M and 98.1% accuracy on ESC-50, setting new benchmarks for audio event detection and classification tasks.
Architecture Overview:
- Audio waveform is converted to a mel spectrogram
- Spectrogram patches are extracted and linearly projected to embedding vectors
- Positional embeddings are added to preserve spatial information
- A stack of Transformer encoder layers processes the patch embeddings
- A classification head maps the aggregated features to event labels
References:
- Paper: “BEATs: Audio Pre-Training with Acoustic Tokenizers” (Chen et al., ICML 2023)
- Repository:
https://github.com/microsoft/unilm/tree/master/beats
Constructors
| Constructor | Summary |
|---|---|
BEATsOptions() |
Initializes BEATs options with the paper defaults. |
BEATsOptions(BEATsOptions other) |
Initializes an independent copy of an existing configuration. |
Properties
| Property | Summary |
|---|---|
OnnxModelOptions OnnxOptions { get; set; } |
Gets or sets ONNX runtime options. |
double AttentionDropoutRate { get; set; } |
Gets or sets the attention dropout rate. |
double DropoutRate { get; set; } |
Gets or sets the dropout rate for regularization. |
double LabelSmoothing { get; set; } |
Gets or sets the label smoothing factor for classification. |
double LearningRate { get; set; } |
Gets or sets the initial learning rate for training. |
double MaskProbability { get; set; } |
Gets or sets the mask probability for masked patch prediction pre-training. |
double Threshold { get; set; } |
Gets or sets the confidence threshold for event detection. |
double WindowOverlap { get; set; } |
Gets or sets the window overlap ratio (0-1) for event detection. |
double WindowSize { get; set; } |
Gets or sets the window size in seconds for event detection. |
int CodebookSize { get; set; } |
Gets or sets the codebook size for the acoustic tokenizer. |
int EmbeddingDim { get; set; } |
Gets or sets the embedding dimension of the Transformer encoder. |
int FMax { get; set; } |
Gets or sets the maximum frequency for the mel filterbank in Hz. |
int FMin { get; set; } |
Gets or sets the minimum frequency for the mel filterbank in Hz. |
int FeedForwardDim { get; set; } |
Gets or sets the feed-forward network dimension in each Transformer layer. |
int FftSize { get; set; } |
Gets or sets the FFT window size for spectrogram computation. |
int HopLength { get; set; } |
Gets or sets the hop length between consecutive FFT frames. |
int MinMaskSpanLength { get; set; } |
Gets or sets the minimum span length for contiguous masking. |
int NumAttentionHeads { get; set; } |
Gets or sets the number of attention heads in each Transformer layer. |
int NumEncoderLayers { get; set; } |
Gets or sets the number of Transformer encoder layers. |
int NumMels { get; set; } |
Gets or sets the number of mel filterbank channels. |
int PatchSize { get; set; } |
Gets or sets the patch size (height) for spectrogram patching. |
int PatchStride { get; set; } |
Gets or sets the patch stride for spectrogram patching. |
int SampleRate { get; set; } |
Gets or sets the audio sample rate in Hz. |
int TokenizerIterations { get; set; } |
Gets or sets the number of acoustic tokenizer iterations. |
int WarmUpSteps { get; set; } |
Gets or sets the number of warm-up steps for learning rate scheduling. |
string ModelPath { get; set; } |
Gets or sets the path to a pre-trained ONNX model file. |
string[] CustomLabels { get; set; } |
Gets or sets custom event labels. |