BEATsOptions

Options & Configuration · AiDotNet.Audio.Classification

Configuration options for the BEATs (Audio Pre-Training with Acoustic Tokenizers) model.

Inheritance: ModelOptions → BEATsOptions

For Beginners

BEATs is a powerful model for recognizing sounds in audio. It works by:

  • Breaking audio into small patches (like cutting a spectrogram into tiles)
  • Using a Transformer (the same architecture behind ChatGPT) to understand relationships between patches
  • Learning what sounds are present by comparing against known audio patterns

You can use BEATs in two ways:

  • ONNX mode: Load a pre-trained model for instant inference
  • Native mode: Train from scratch on your own audio data

Example usage:

How It Works

BEATs (Chen et al., ICML 2023) is a state-of-the-art audio classification model that uses iterative self-distillation between an acoustic tokenizer and an audio SSL model. It achieves 50.6% mAP on AudioSet-2M and 98.1% accuracy on ESC-50, setting new benchmarks for audio event detection and classification tasks.

Architecture Overview:

  • Audio waveform is converted to a mel spectrogram
  • Spectrogram patches are extracted and linearly projected to embedding vectors
  • Positional embeddings are added to preserve spatial information
  • A stack of Transformer encoder layers processes the patch embeddings
  • A classification head maps the aggregated features to event labels

References:

  • Paper: “BEATs: Audio Pre-Training with Acoustic Tokenizers” (Chen et al., ICML 2023)
  • Repository: https://github.com/microsoft/unilm/tree/master/beats

Constructors

Constructor Summary
BEATsOptions() Initializes BEATs options with the paper defaults.
BEATsOptions(BEATsOptions other) Initializes an independent copy of an existing configuration.

Properties

Property Summary
OnnxModelOptions OnnxOptions { get; set; } Gets or sets ONNX runtime options.
double AttentionDropoutRate { get; set; } Gets or sets the attention dropout rate.
double DropoutRate { get; set; } Gets or sets the dropout rate for regularization.
double LabelSmoothing { get; set; } Gets or sets the label smoothing factor for classification.
double LearningRate { get; set; } Gets or sets the initial learning rate for training.
double MaskProbability { get; set; } Gets or sets the mask probability for masked patch prediction pre-training.
double Threshold { get; set; } Gets or sets the confidence threshold for event detection.
double WindowOverlap { get; set; } Gets or sets the window overlap ratio (0-1) for event detection.
double WindowSize { get; set; } Gets or sets the window size in seconds for event detection.
int CodebookSize { get; set; } Gets or sets the codebook size for the acoustic tokenizer.
int EmbeddingDim { get; set; } Gets or sets the embedding dimension of the Transformer encoder.
int FMax { get; set; } Gets or sets the maximum frequency for the mel filterbank in Hz.
int FMin { get; set; } Gets or sets the minimum frequency for the mel filterbank in Hz.
int FeedForwardDim { get; set; } Gets or sets the feed-forward network dimension in each Transformer layer.
int FftSize { get; set; } Gets or sets the FFT window size for spectrogram computation.
int HopLength { get; set; } Gets or sets the hop length between consecutive FFT frames.
int MinMaskSpanLength { get; set; } Gets or sets the minimum span length for contiguous masking.
int NumAttentionHeads { get; set; } Gets or sets the number of attention heads in each Transformer layer.
int NumEncoderLayers { get; set; } Gets or sets the number of Transformer encoder layers.
int NumMels { get; set; } Gets or sets the number of mel filterbank channels.
int PatchSize { get; set; } Gets or sets the patch size (height) for spectrogram patching.
int PatchStride { get; set; } Gets or sets the patch stride for spectrogram patching.
int SampleRate { get; set; } Gets or sets the audio sample rate in Hz.
int TokenizerIterations { get; set; } Gets or sets the number of acoustic tokenizer iterations.
int WarmUpSteps { get; set; } Gets or sets the number of warm-up steps for learning rate scheduling.
string ModelPath { get; set; } Gets or sets the path to a pre-trained ONNX model file.
string[] CustomLabels { get; set; } Gets or sets custom event labels.