DocFormerSpatialEmbeddingLayer<T>
Layers · AiDotNet.NeuralNetworks.Layers
DocFormer’s spatial embedding of one stream (Appalaraju et al., ICCV 2021; reference DocFormerEmbeddings).
Inheritance: LayerBase<T> → DocFormerSpatialEmbeddingLayer<T>
Implements: IShapeContract
For Beginners
This turns each word’s box, and where it sits relative to the word before it, into a vector the attention can compare.
How It Works
Per axis, a token is described by eight lookups:
- Its two edges (tables of width
coordinateSize). - Its extent.
- The distances of its four corners and its centroid to the previous token’s (tables of width
coordinateSize; distances are clamped to +-maxPosition2D).
The eight are concatenated. The x and y concatenations and a sinusoidal position are then summed, giving
8 x coordinateSize = hidden. The visual and text streams each own one of these layers, as in the
reference.
Input: boxes [S, 4] as integer (x0, y0, x1, y1) on the 0 .. maxPosition2D - 1 grid.
Constructors
| Constructor | Summary |
|---|---|
DocFormerSpatialEmbeddingLayer(int hidden, int maxPosition2D) |
Creates the embedding. |
Properties
| Property | Summary |
|---|---|
IReadOnlyList<LayerPort> InputPorts { get; } |
Generated from [TensorPort] declarations. |
IReadOnlyList<LayerPort> OutputPorts { get; } |
Generated from [TensorPort] declarations. |
bool SupportsTraining { get; } |
Methods
| Method | Summary |
|---|---|
IReadOnlyList<ILayer<T>> GetSubLayers() |
|
IReadOnlyList<OutputAxisContract> OutputAxesFor(int inputRank) |
|
LayerInputDomain GetInputDomain(int[] inputShape) |
Generated from the primary [TensorPort] declaration. |
void ResetState() |