EfficientTAMOptions
Options & Configuration · AiDotNet.ComputerVision.Segmentation.Video
Configuration options for EfficientTAM, Efficient Track Anything (Xiong et al. 2024): SAM 2’s tracker on
a plain ViT image encoder, optionally with the efficient (2×2 pooled) memory cross-attention.
Inheritance: ModelOptions → NeuralNetworkOptions → ModelHyperparameterOptions → SegmentationModelOptions → EfficientTAMOptions
For Beginners
Choose ModelSize and, for the faster “/2” models,
MemoryCrossAttention; leave the rest at its defaults.
How It Works
The defaults are EfficientTAM-S (ViT-Small, SAM 2’s memory block) at 1024 pixels and the paper’s full training (Table
in section 4): AdamW (0.9, 0.999), 6e-5 for the image encoder and 3e-4 for the rest with layer-wise decay 0.8, a
15k-step linear warmup into cosine decay over 300k steps, weight decay 0.1, no drop path, and the focal (20), dice (1),
ℓ1 IoU (1) and occlusion (1) losses. Where the paper is silent (the gradient clip, the decay exemptions, the prompt
simulation) SAM 2’s recipe, which EfficientTAM follows, fills in. MemoryCrossAttention selects the “/2”
models’ efficient cross-attention.
Constructors
| Constructor | Summary |
|---|---|
EfficientTAMOptions() |
Initializes a new instance with default values. |
EfficientTAMOptions(EfficientTAMOptions other) |
Initializes a new instance by copying from another instance. |
Properties
| Property | Summary |
|---|---|
EfficientTAMModelSize ModelSize { get; set; } |
The image encoder: ViT-Tiny or ViT-Small (default: Small). |
MemoryCrossAttention MemoryCrossAttention { get; set; } |
How the memory attention reads the spatial memories: SAM 2’s full attention (EfficientTAM-Ti and -S, the default), or the efficient 2×2 pooled cross-attention of the “/2” models (eq. |
double FocalAlpha { get; set; } |
Focal loss’s α (0.25). |
double FocalGamma { get; set; } |
Focal loss’s γ (2). |
double ImageEncoderLearningRate { get; set; } |
The image encoder’s learning rate (paper: 6e-5). |
double IouLossWeight { get; set; } |
The ℓ1 IoU-loss weight (paper: 1). |
double LearningRate { get; set; } |
The learning rate of everything but the image encoder (paper: 3e-4). |
double MaskDiceWeight { get; set; } |
The dice-loss weight (paper: 1). |
double MaskFocalWeight { get; set; } |
The focal-loss weight (paper: 20). |
double MaxGradientNorm { get; set; } |
The gradient-norm clip (SAM 2’s 0.1; the paper gives none). |
double OcclusionLossWeight { get; set; } |
The occlusion cross-entropy weight (paper: 1). |
double WeightDecay { get; set; } |
AdamW’s decoupled weight decay (paper: 0.1). |
double? DropPath { get; set; } |
The largest stochastic-depth rate (paper: none). |
double? LayerDecay { get; set; } |
The per-block layer-wise decay of the ViT’s learning rates (paper: 0.8). |
int CorrectionPoints { get; set; } |
Corrective clicks per corrected training frame (SAM 2’s 7). |
int MemoryBankSize { get; set; } |
The memory bank: the prompted frames and up to MemoryBankSize − 1 recent frames (default: 7). |
int TrainingSteps { get; set; } |
The training steps the cosine schedule spans (paper: 300k). |
int WarmupSteps { get; set; } |
The linear warmup steps (paper: 15k). |
int? DecoderHeadCount { get; set; } |
The mask decoder’s attention heads (null: 8). |
int? ImageSize { get; set; } |
The square size frames are resized to for the image encoder (null: the architecture’s larger side; the released models use 1024). |
int? MaskDecoderMlpDimension { get; set; } |
The mask decoder’s MLP width (null: 2048). |
int? MemoryAttentionFeedForwardDimension { get; set; } |
The memory attention’s feed-forward width (null: 2048). |
int? MemoryAttentionLayerCount { get; set; } |
The memory attention layers (null: 4). |
int? MemoryDimension { get; set; } |
The memory width (null: 64). |
int? MemoryFuserDimension { get; set; } |
The memory fuser’s hidden width (null: 1024). |
int? ModelDimension { get; set; } |
The prompt, mask-decoder and memory-attention width (null: 256). |
int? VitDepth { get; set; } |
The ViT’s blocks (null: 12). |
int? VitEmbeddingDimension { get; set; } |
The ViT’s width (null: Tiny 192, Small 384). |
int? VitHeadCount { get; set; } |
The ViT’s attention heads (null: Tiny 3, Small 6). |
int? VitWindowSize { get; set; } |
The ViT’s attention window (null: 14). |
int[] VitGlobalAttentionBlockIndexes { get; set; } |
The ViT’s global-attention blocks (null: 2, 5, 8, 11). |
Methods
| Method | Summary |
|---|---|
void Validate() |
Throws if a setting is out of range. |