SamViTDetEncoderLayer<T>
Layers ยท AiDotNet.NeuralNetworks.Layers
The SAM ViTDet image encoder that GOT-OCR2 uses (Kirillov et al. 2023; Wei et al. 2024; HF
GotOcr2VisionEncoder).
Inheritance: LayerBase<T> โ SamViTDetEncoderLayer<T>
Implements: IShapeContract
How It Works
- A
patchSize-stride patch convolution, then an absolute position table. - Pre-LN blocks
x += attn(LN x); x += MLP(LN x). Attention is windowed, except in every
globalEvery-th block, which attends over the whole grid. For ViTDet-B that is blocks 2, 5, 8 and
11.
- Each block adds decomposed relative positions (Li et al. 2022, MViTv2). The bias between query
(qh, qw) and key (kh, kw) is q . Rh[qh - kh] + q . Rw[qw - kw], using the unscaled query.
- A neck of conv1x1, LN2d, conv3x3, LN2d down to
neckChannels, with bias-free convolutions.
The position and relative tables start at zero, as the reference initialises them.
Constructors
| Constructor | Summary |
|---|---|
SamViTDetEncoderLayer(int imageSize, int patchSize, int dim, int numLayers, int numHeads, int mlpDim, int windowSize, int globalEvery, int neckChannels) |
Properties
| Property | Summary |
|---|---|
bool SupportsTraining { get; } |
|
int Grid { get; } |
Side of the output map. |
int NeckChannels { get; } |
Channels of the output map. |
Methods
| Method | Summary |
|---|---|
Dictionary<string, string> GetParameterRoles() |
Returns parameter roles for per-role optimizer configuration (e.g., weight decay exemption for biases). |
IReadOnlyList<ILayer<T>> GetSubLayers() |
|
IReadOnlyList<OutputAxisContract> OutputAxesFor(int inputRank) |
|
IReadOnlyList<Tensor<T>> GetTrainableParameters() |
Returns all trainable parameter tensors marked with [TrainableParameter]. |
void ResetState() |
|
void SetTrainableParameters(IReadOnlyList<Tensor<T>> parameters) |
Replaces trainable parameter tensors (e.g., with ParameterBuffer views). |
void ZeroGrad() |
Clears all gradient fields discovered by convention ({paramName}Gradient). |