CLAPModelOptions
Options & Configuration · AiDotNet.Models.Options
Configuration options for CLAP (Contrastive Language-Audio Pretraining) models.
Inheritance: ModelOptions → NeuralNetworkOptions → ModelHyperparameterOptions → AudioNeuralNetworkOptions → CLAPModelOptions
For Beginners
These knobs control how big and deep the audio and text encoders are. The defaults match the published CLAP checkpoint and produce 512-dim contrastive embeddings. Override individual fields to build a smaller / faster variant.
How It Works
Defaults are LAION-CLAP’s published “HTSAT-tiny + RoBERTa” model with feature fusion (Wu et al. 2023,
“Large-Scale Contrastive Language-Audio Pre-Training with Feature Fusion and Keyword-to-Caption Augmentation”;
reference LAION-AI/CLAP, model_configs/HTSAT-tiny.json and htsat.create_htsat_model):
a 48 kHz, 64-band log-mel front end from 50 Hz to 14 kHz over 10-second clips; the HTS-AT audio encoder
(Chen et al. 2022), a four-stage hierarchical Swin Transformer; the RoBERTa-base text encoder (Liu et al.
2019); and Linear → ReLU → Linear projections into a 512-dimensional joint space.
Constructors
| Constructor | Summary |
|---|---|
CLAPModelOptions() |
Initializes a new instance with the LAION-CLAP HTSAT-tiny + RoBERTa-base defaults. |
CLAPModelOptions(CLAPModelOptions other) |
Initializes a new instance by copying every property from other. |
Properties
| Property | Summary |
|---|---|
bool EnableFeatureFusion { get; set; } |
Whether long clips use feature fusion (Wu et al. |
double AudioDropPathRate { get; set; } |
Stochastic-depth rate of the last Swin block; the rate grows linearly from 0 across blocks. |
double InitialTemperature { get; set; } |
Initial temperature τ for the contrastive softmax; the model learns log(1/τ). |
double MelMaxFrequency { get; set; } |
Highest frequency of the mel filterbank, in Hz. |
double MelMinFrequency { get; set; } |
Lowest frequency of the mel filterbank, in Hz. |
double TextDropoutRate { get; set; } |
Hidden and attention-probability dropout of the text encoder. |
double TextLayerNormEpsilon { get; set; } |
LayerNorm epsilon of the text encoder. |
int AudioEmbedDim { get; set; } |
Embedding width of the first stage; it doubles at every patch merging. |
int AudioMlpRatio { get; set; } |
Ratio of the Swin MLP’s hidden width to the block width. |
int AudioPatchSize { get; set; } |
Patch size (and stride) of the patch embedding. |
int ClipSamples { get; set; } |
Samples per clip: shorter audio is repeated and zero-padded to this length, longer audio is cropped (no fusion) or fused (feature fusion). |
int HopLength { get; set; } |
STFT hop length in samples. |
int MaxTextLength { get; set; } |
Token length captions are truncated / padded to. |
int NumMelBands { get; set; } |
Number of mel-frequency bands. |
int ProjectionDim { get; set; } |
Dimension of the joint audio-text embedding space. |
int SampleRate { get; set; } |
Audio sample rate in Hz the encoder is configured for. |
int SpecAugmentFreqDropWidth { get; set; } |
Exclusive upper bound of the width, in mel bins, of a frequency stripe zeroed during training. |
int SpecAugmentFreqStripes { get; set; } |
Frequency stripes zeroed per training sample; 0 turns frequency masking off. |
int SpecAugmentTimeDropWidth { get; set; } |
Exclusive upper bound of the width, in frames, of a time stripe zeroed during training. |
int SpecAugmentTimeStripes { get; set; } |
Time stripes zeroed per training sample; 0 turns time masking off. |
int SpecSize { get; set; } |
Side of the square spectrogram image HTS-AT reads. |
int StftWindowSize { get; set; } |
STFT window (and FFT) size in samples. |
int SwinWindowSize { get; set; } |
Swin window size (W-MSA / SW-MSA). |
int TextEncoderHeads { get; set; } |
Attention heads per text encoder layer. |
int TextEncoderLayers { get; set; } |
Number of text encoder layers. |
int TextHiddenDim { get; set; } |
Hidden width of the text encoder. |
int TextIntermediateDim { get; set; } |
Width of the text encoder’s feed-forward intermediate projection. |
int TextMaxPositions { get; set; } |
Rows of RoBERTa’s position table (positions start after the padding id). |
int TextPaddingTokenId { get; set; } |
RoBERTa’s padding token id (<pad>). |
int VocabSize { get; set; } |
Text vocabulary size — RoBERTa BPE vocab. |
int[] AudioStageDepths { get; set; } |
Swin blocks per stage; one stage per entry, with a patch merging between consecutive stages. |
int[] AudioStageHeads { get; set; } |
Attention heads per stage; one entry per stage. |
string TextEncoderPath { get; set; } |
Gets or sets the text encoder path. |
Methods
| Method | Summary |
|---|---|
void Validate() |
Throws if a value this model requires has been left unset or is inconsistent. |