SAM2Options

Options & Configuration · AiDotNet.Video.Options

Configuration options for SAM2, SAM 2 (Ravi et al. 2024) for images and videos.

Inheritance: ModelOptions → NeuralNetworkOptions → ModelHyperparameterOptions → VideoHyperparameterOptions → SAM2Options

For Beginners

You only need ModelSize; leave everything else at its default unless you are building a smaller model to experiment with.

How It Works

The defaults are the released SAM 2 Hiera-B+ model at 1024 pixels and the paper’s full-training recipe (Table 12b): AdamW with weight decay 0.1 (not on biases and LayerNorms, as the official config has it), learning rates 3e-4 and 6e-5 for the image encoder with layer-wise decay, linear warmup over 7.5k steps then cosine decay (to a tenth, as the official config ends it), gradient clipping at 0.1, and the focal (20), dice (1), ℓ1 IoU (1) and occlusion cross-entropy (1) losses.

The Hiera…, ModelDimension and memory settings are capacity overrides: left null, every one follows the selected ModelSize, so the model can load that size’s released checkpoint.

Constructors

Constructor Summary
SAM2Options()

Properties

Property Summary
SAM2ModelSize ModelSize { get; set; } The released model size (default: Base+, Base).
bool Sam21 { get; set; } Whether to build SAM 2.1 rather than SAM 2: the same network, with the temporal encoding of the object pointers and the occlusion embedding of the memories that SAM 2.1 adds (default: false).
double FocalAlpha { get; set; } Focal loss’s α (0.25, the RetinaNet value SAM uses).
double FocalGamma { get; set; } Focal loss’s γ (2, the RetinaNet value SAM uses).
double ImageEncoderLearningRate { get; set; } The image encoder’s learning rate (paper: 6e-5).
double IouLossWeight { get; set; } The ℓ1 IoU-loss weight (paper: 1).
double LearningRate { get; set; } The learning rate of everything but the image encoder (paper: 3e-4).
double MaskDiceWeight { get; set; } The dice-loss weight (paper: 1).
double MaskFocalWeight { get; set; } The focal-loss weight (paper: 20).
double MaxGradientNorm { get; set; } The gradient-norm clip (paper: 0.1).
double OcclusionLossWeight { get; set; } The occlusion cross-entropy weight (paper: 1).
double WeightDecay { get; set; } AdamW’s decoupled weight decay (paper: 0.1).
double? DropPath { get; set; } The largest stochastic-depth rate of the Hiera blocks (null: the paper’s 0.1 for Tiny and Small, 0.2 for Base+, 0.3 for Large).
double? LayerDecay { get; set; } The per-block layer-wise decay of the Hiera learning rates (null: the paper’s 0.8 for Tiny and Small, 0.9 for Base+, 0.925 for Large).
int CorrectionPoints { get; set; } Corrective clicks per corrected training frame (paper: 7).
int MemoryBankSize { get; set; } The memory bank: the prompted frames and up to MemoryBankSize − 1 recent frames (default: 7).
int TrainingSteps { get; set; } The training steps the cosine schedule spans (paper: about 150k).
int WarmupSteps { get; set; } The linear warmup steps (paper: 7.5k).
int? DecoderHeadCount { get; set; } The mask decoder’s attention heads (null: 8).
int? HieraEmbeddingDimension { get; set; } The Hiera stem width (null: Tiny and Small 96, Base+ 112, Large 144); it doubles per stage.
int? HieraInitialHeadCount { get; set; } The first stage’s attention heads (null: Tiny and Small 1, Base+ and Large 2); they double per stage.
int? ImageSize { get; set; } The square size frames are resized to for the image encoder (null: the architecture’s larger side; the released models use 1024).
int? MaskDecoderMlpDimension { get; set; } The mask decoder’s MLP width (null: 2048).
int? MemoryAttentionFeedForwardDimension { get; set; } The memory attention’s feed-forward width (null: 2048).
int? MemoryAttentionLayerCount { get; set; } The memory attention layers (null: 4).
int? MemoryDimension { get; set; } The memory width (null: 64).
int? MemoryFuserDimension { get; set; } The memory fuser’s hidden width (null: 1024).
int? ModelDimension { get; set; } The FPN, prompt and mask-decoder width (null: 256).
int[] HieraGlobalAttentionBlockIndexes { get; set; } The blocks (0-based over all stages) with global attention (null: the size’s).
int[] HieraStageDepths { get; set; } The Hiera blocks per stage (four values; null: the size’s).
int[] HieraWindowSizes { get; set; } The windowed-attention size per stage (four values; null: the size’s).

Methods

Method Summary
void Validate() Throws if a setting is out of range.