SAM2Options
Options & Configuration · AiDotNet.Video.Options
Configuration options for SAM2, SAM 2 (Ravi et al. 2024) for images and videos.
Inheritance: ModelOptions → NeuralNetworkOptions → ModelHyperparameterOptions → VideoHyperparameterOptions → SAM2Options
For Beginners
You only need ModelSize; leave everything else at its default unless you are
building a smaller model to experiment with.
How It Works
The defaults are the released SAM 2 Hiera-B+ model at 1024 pixels and the paper’s full-training recipe (Table 12b): AdamW with weight decay 0.1 (not on biases and LayerNorms, as the official config has it), learning rates 3e-4 and 6e-5 for the image encoder with layer-wise decay, linear warmup over 7.5k steps then cosine decay (to a tenth, as the official config ends it), gradient clipping at 0.1, and the focal (20), dice (1), ℓ1 IoU (1) and occlusion cross-entropy (1) losses.
The Hiera…, ModelDimension and memory settings are capacity overrides: left null, every one follows the
selected ModelSize, so the model can load that size’s released checkpoint.
Constructors
| Constructor | Summary |
|---|---|
SAM2Options() |
Properties
| Property | Summary |
|---|---|
SAM2ModelSize ModelSize { get; set; } |
The released model size (default: Base+, Base). |
bool Sam21 { get; set; } |
Whether to build SAM 2.1 rather than SAM 2: the same network, with the temporal encoding of the object pointers and the occlusion embedding of the memories that SAM 2.1 adds (default: false). |
double FocalAlpha { get; set; } |
Focal loss’s α (0.25, the RetinaNet value SAM uses). |
double FocalGamma { get; set; } |
Focal loss’s γ (2, the RetinaNet value SAM uses). |
double ImageEncoderLearningRate { get; set; } |
The image encoder’s learning rate (paper: 6e-5). |
double IouLossWeight { get; set; } |
The ℓ1 IoU-loss weight (paper: 1). |
double LearningRate { get; set; } |
The learning rate of everything but the image encoder (paper: 3e-4). |
double MaskDiceWeight { get; set; } |
The dice-loss weight (paper: 1). |
double MaskFocalWeight { get; set; } |
The focal-loss weight (paper: 20). |
double MaxGradientNorm { get; set; } |
The gradient-norm clip (paper: 0.1). |
double OcclusionLossWeight { get; set; } |
The occlusion cross-entropy weight (paper: 1). |
double WeightDecay { get; set; } |
AdamW’s decoupled weight decay (paper: 0.1). |
double? DropPath { get; set; } |
The largest stochastic-depth rate of the Hiera blocks (null: the paper’s 0.1 for Tiny and Small, 0.2 for Base+, 0.3 for Large). |
double? LayerDecay { get; set; } |
The per-block layer-wise decay of the Hiera learning rates (null: the paper’s 0.8 for Tiny and Small, 0.9 for Base+, 0.925 for Large). |
int CorrectionPoints { get; set; } |
Corrective clicks per corrected training frame (paper: 7). |
int MemoryBankSize { get; set; } |
The memory bank: the prompted frames and up to MemoryBankSize − 1 recent frames (default: 7). |
int TrainingSteps { get; set; } |
The training steps the cosine schedule spans (paper: about 150k). |
int WarmupSteps { get; set; } |
The linear warmup steps (paper: 7.5k). |
int? DecoderHeadCount { get; set; } |
The mask decoder’s attention heads (null: 8). |
int? HieraEmbeddingDimension { get; set; } |
The Hiera stem width (null: Tiny and Small 96, Base+ 112, Large 144); it doubles per stage. |
int? HieraInitialHeadCount { get; set; } |
The first stage’s attention heads (null: Tiny and Small 1, Base+ and Large 2); they double per stage. |
int? ImageSize { get; set; } |
The square size frames are resized to for the image encoder (null: the architecture’s larger side; the released models use 1024). |
int? MaskDecoderMlpDimension { get; set; } |
The mask decoder’s MLP width (null: 2048). |
int? MemoryAttentionFeedForwardDimension { get; set; } |
The memory attention’s feed-forward width (null: 2048). |
int? MemoryAttentionLayerCount { get; set; } |
The memory attention layers (null: 4). |
int? MemoryDimension { get; set; } |
The memory width (null: 64). |
int? MemoryFuserDimension { get; set; } |
The memory fuser’s hidden width (null: 1024). |
int? ModelDimension { get; set; } |
The FPN, prompt and mask-decoder width (null: 256). |
int[] HieraGlobalAttentionBlockIndexes { get; set; } |
The blocks (0-based over all stages) with global attention (null: the size’s). |
int[] HieraStageDepths { get; set; } |
The Hiera blocks per stage (four values; null: the size’s). |
int[] HieraWindowSizes { get; set; } |
The windowed-attention size per stage (four values; null: the size’s). |
Methods
| Method | Summary |
|---|---|
void Validate() |
Throws if a setting is out of range. |