SamViTDetEncoderLayer<T>

Layers ยท AiDotNet.NeuralNetworks.Layers

The SAM ViTDet image encoder that GOT-OCR2 uses (Kirillov et al. 2023; Wei et al. 2024; HF GotOcr2VisionEncoder).

Inheritance: LayerBase<T> โ†’ SamViTDetEncoderLayer<T>

Implements: IShapeContract

How It Works

  • A patchSize-stride patch convolution, then an absolute position table.
  • Pre-LN blocks x += attn(LN x); x += MLP(LN x). Attention is windowed, except in every

globalEvery-th block, which attends over the whole grid. For ViTDet-B that is blocks 2, 5, 8 and 11.

  • Each block adds decomposed relative positions (Li et al. 2022, MViTv2). The bias between query

(qh, qw) and key (kh, kw) is q . Rh[qh - kh] + q . Rw[qw - kw], using the unscaled query.

  • A neck of conv1x1, LN2d, conv3x3, LN2d down to neckChannels, with bias-free convolutions.

The position and relative tables start at zero, as the reference initialises them.

Constructors

Constructor Summary
SamViTDetEncoderLayer(int imageSize, int patchSize, int dim, int numLayers, int numHeads, int mlpDim, int windowSize, int globalEvery, int neckChannels)

Properties

Property Summary
bool SupportsTraining { get; }
int Grid { get; } Side of the output map.
int NeckChannels { get; } Channels of the output map.

Methods

Method Summary
Dictionary<string, string> GetParameterRoles() Returns parameter roles for per-role optimizer configuration (e.g., weight decay exemption for biases).
IReadOnlyList<ILayer<T>> GetSubLayers()
IReadOnlyList<OutputAxisContract> OutputAxesFor(int inputRank)
IReadOnlyList<Tensor<T>> GetTrainableParameters() Returns all trainable parameter tensors marked with [TrainableParameter].
void ResetState()
void SetTrainableParameters(IReadOnlyList<Tensor<T>> parameters) Replaces trainable parameter tensors (e.g., with ParameterBuffer views).
void ZeroGrad() Clears all gradient fields discovered by convention ({paramName}Gradient).