DaViTLayer<T>

Layers ยท AiDotNet.NeuralNetworks.Layers

DaViT, the dual-attention vision transformer that Florence-2 uses as its image encoder (Ding et al. 2022; HF Florence2VisionBackbone).

Inheritance: LayerBase<T> โ†’ DaViTLayer<T>

Implements: IShapeContract

How It Works

There are four stages, and each starts with a strided convolutional patch embedding (kernels 7/3/3/3, strides 4/2/2/2). The total stride is 32. Stage 0 applies its LayerNorm after the convolution; the later stages apply it before.

Every block pairs a spatial block with a channel block. Each has the form x += dwconv(x); x += attention(LN(x)); x += dwconv(x); x += MLP(LN(x)):

Widths and heads (also the group counts) double at every stage from baseDim and baseHeads. Depths are 1/1/thirdStageDepth/1. Florence-2-base is (128, 4, 9) and Florence-2-large is (256, 8, 9). Stochastic depth, a pretraining regularizer, is not applied.

Constructors

Constructor Summary
DaViTLayer(int imageSize, int baseDim, int baseHeads, int thirdStageDepth, int windowSize, int mlpRatio)

Properties

Property Summary
bool SupportsTraining { get; }
int OutputChannels { get; } Channels of the stride-32 output map.
int OutputGrid { get; } Side of the square stride-32 output map.

Methods

Method Summary
IReadOnlyList<ILayer<T>> GetSubLayers()
IReadOnlyList<OutputAxisContract> OutputAxesFor(int inputRank)
void ResetState()