HTSATOptions
Options & Configuration · AiDotNet.Audio.Classification
Configuration options for the HTS-AT (Hierarchical Token-Semantic Audio Transformer) model.
Inheritance: ModelOptions → HTSATOptions
For Beginners
HTS-AT is an efficient audio classifier that processes spectrograms hierarchically (like zooming from overview to detail). It uses a technique called “window attention” that looks at local regions first, then gradually combines them. This makes it faster and more memory-efficient than models like AST that look at everything at once.
How It Works
HTS-AT (Chen et al., ICASSP 2022) is a hierarchical Transformer architecture for audio classification that uses Swin Transformer blocks with token-semantic modules to efficiently process audio spectrograms. It achieves 47.1% mAP on AudioSet-2M with only 30M parameters.
References:
- Paper: “HTS-AT: A Hierarchical Token-Semantic Audio Transformer” (Chen et al., ICASSP 2022)
- Repository:
https://github.com/RetroCirce/HTS-Audio-Transformer
Constructors
| Constructor | Summary |
|---|---|
HTSATOptions() |
Properties
| Property | Summary |
|---|---|
OnnxModelOptions OnnxOptions { get; set; } |
Gets or sets ONNX runtime options. |
bool UseTokenSemanticModule { get; set; } |
Gets or sets whether to use the token-semantic module. |
double AttentionDropoutRate { get; set; } |
Gets or sets the attention dropout rate. |
double DetectionWindowSize { get; set; } |
Gets or sets the window size in seconds for event detection. |
double DropPathRate { get; set; } |
Gets or sets the drop path rate for stochastic depth. |
double DropoutRate { get; set; } |
Gets or sets the dropout rate. |
double FeedForwardRatio { get; set; } |
Gets or sets the feed-forward expansion ratio. |
double LabelSmoothing { get; set; } |
Gets or sets the label smoothing factor. |
double LearningRate { get; set; } |
Gets or sets the initial learning rate. |
double Threshold { get; set; } |
Gets or sets the confidence threshold for event detection. |
double WindowOverlap { get; set; } |
Gets or sets the window overlap ratio (0-1). |
int EmbeddingDim { get; set; } |
Gets or sets the embedding dimension. |
int FMax { get; set; } |
Gets or sets the maximum frequency for mel filterbank. |
int FMin { get; set; } |
Gets or sets the minimum frequency for mel filterbank. |
int FftSize { get; set; } |
Gets or sets the FFT window size. |
int HopLength { get; set; } |
Gets or sets the hop length between FFT frames. |
int NumMels { get; set; } |
Gets or sets the number of mel filterbank channels. |
int NumSemanticGroups { get; set; } |
Gets or sets the number of semantic groups. |
int PatchSize { get; set; } |
Gets or sets the patch size for initial embedding. |
int SampleRate { get; set; } |
Gets or sets the audio sample rate in Hz. |
int WarmUpSteps { get; set; } |
Gets or sets the warm-up steps. |
int WindowSize { get; set; } |
Gets or sets the window size for local attention. |
int[] NumHeadsPerStage { get; set; } |
Gets or sets the number of attention heads per stage. |
int[] NumLayersPerStage { get; set; } |
Gets or sets the number of Swin Transformer layers per stage. |
string ModelPath { get; set; } |
Gets or sets the path to a pre-trained ONNX model file. |
string[] CustomLabels { get; set; } |
Gets or sets custom event labels. |