CLAPModelOptions

Options & Configuration · AiDotNet.Models.Options

Configuration options for CLAP (Contrastive Language-Audio Pretraining) models.

Inheritance: ModelOptions → NeuralNetworkOptions → ModelHyperparameterOptions → AudioNeuralNetworkOptions → CLAPModelOptions

For Beginners

These knobs control how big and deep the audio and text encoders are. The defaults match the published CLAP checkpoint and produce 512-dim contrastive embeddings. Override individual fields to build a smaller / faster variant.

How It Works

Defaults are LAION-CLAP’s published “HTSAT-tiny + RoBERTa” model with feature fusion (Wu et al. 2023, “Large-Scale Contrastive Language-Audio Pre-Training with Feature Fusion and Keyword-to-Caption Augmentation”; reference LAION-AI/CLAP, model_configs/HTSAT-tiny.json and htsat.create_htsat_model): a 48 kHz, 64-band log-mel front end from 50 Hz to 14 kHz over 10-second clips; the HTS-AT audio encoder (Chen et al. 2022), a four-stage hierarchical Swin Transformer; the RoBERTa-base text encoder (Liu et al. 2019); and Linear → ReLU → Linear projections into a 512-dimensional joint space.

Constructors

Constructor Summary
CLAPModelOptions() Initializes a new instance with the LAION-CLAP HTSAT-tiny + RoBERTa-base defaults.
CLAPModelOptions(CLAPModelOptions other) Initializes a new instance by copying every property from other.

Properties

Property Summary
bool EnableFeatureFusion { get; set; } Whether long clips use feature fusion (Wu et al.
double AudioDropPathRate { get; set; } Stochastic-depth rate of the last Swin block; the rate grows linearly from 0 across blocks.
double InitialTemperature { get; set; } Initial temperature τ for the contrastive softmax; the model learns log(1/τ).
double MelMaxFrequency { get; set; } Highest frequency of the mel filterbank, in Hz.
double MelMinFrequency { get; set; } Lowest frequency of the mel filterbank, in Hz.
double TextDropoutRate { get; set; } Hidden and attention-probability dropout of the text encoder.
double TextLayerNormEpsilon { get; set; } LayerNorm epsilon of the text encoder.
int AudioEmbedDim { get; set; } Embedding width of the first stage; it doubles at every patch merging.
int AudioMlpRatio { get; set; } Ratio of the Swin MLP’s hidden width to the block width.
int AudioPatchSize { get; set; } Patch size (and stride) of the patch embedding.
int ClipSamples { get; set; } Samples per clip: shorter audio is repeated and zero-padded to this length, longer audio is cropped (no fusion) or fused (feature fusion).
int HopLength { get; set; } STFT hop length in samples.
int MaxTextLength { get; set; } Token length captions are truncated / padded to.
int NumMelBands { get; set; } Number of mel-frequency bands.
int ProjectionDim { get; set; } Dimension of the joint audio-text embedding space.
int SampleRate { get; set; } Audio sample rate in Hz the encoder is configured for.
int SpecAugmentFreqDropWidth { get; set; } Exclusive upper bound of the width, in mel bins, of a frequency stripe zeroed during training.
int SpecAugmentFreqStripes { get; set; } Frequency stripes zeroed per training sample; 0 turns frequency masking off.
int SpecAugmentTimeDropWidth { get; set; } Exclusive upper bound of the width, in frames, of a time stripe zeroed during training.
int SpecAugmentTimeStripes { get; set; } Time stripes zeroed per training sample; 0 turns time masking off.
int SpecSize { get; set; } Side of the square spectrogram image HTS-AT reads.
int StftWindowSize { get; set; } STFT window (and FFT) size in samples.
int SwinWindowSize { get; set; } Swin window size (W-MSA / SW-MSA).
int TextEncoderHeads { get; set; } Attention heads per text encoder layer.
int TextEncoderLayers { get; set; } Number of text encoder layers.
int TextHiddenDim { get; set; } Hidden width of the text encoder.
int TextIntermediateDim { get; set; } Width of the text encoder’s feed-forward intermediate projection.
int TextMaxPositions { get; set; } Rows of RoBERTa’s position table (positions start after the padding id).
int TextPaddingTokenId { get; set; } RoBERTa’s padding token id (<pad>).
int VocabSize { get; set; } Text vocabulary size — RoBERTa BPE vocab.
int[] AudioStageDepths { get; set; } Swin blocks per stage; one stage per entry, with a patch merging between consecutive stages.
int[] AudioStageHeads { get; set; } Attention heads per stage; one entry per stage.
string TextEncoderPath { get; set; } Gets or sets the text encoder path.

Methods

Method Summary
void Validate() Throws if a value this model requires has been left unset or is inconsistent.