LayoutEmbeddingLayer<T>
Layers · AiDotNet.NeuralNetworks.Layers
Paper-faithful text + 2D-layout embedding front-end for layout-aware document models (Xu et al., KDD 2020, “LayoutLM: Pre-training of Text and Layout for Document Image Understanding”, §3.2 “Model Architecture”).
Inheritance: LayerBase<T> → LayoutEmbeddingLayer<T>
Implements: IShapeContract
For Beginners
A plain language model sees a document as one long line of words, so it cannot tell a table cell from a footnote. This layer hands the model the coordinates of the box each word was printed in, so “500” in the top-right corner of an invoice can be learned to mean something different from “500” in the body text. You give it the word IDs and, optionally, the boxes; it gives back one vector per word that mixes both.
How It Works
LayoutLM’s contribution over BERT is that a token carries WHERE it sits on the page, not just what it says. The paper builds one embedding block that sums a token’s identity, its position in the reading order, and six numbers derived from its bounding box:
The x-axis table is SHARED between the left and right edges, and the y-axis table between the top and bottom edges — the paper uses “two embedding tables to embed x-axis and y-axis features separately”, so a coordinate means the same thing whichever corner it came from. Width and height get their own tables because a box’s SIZE is a different feature from its POSITION: two boxes can start at the same x and be a caption and a paragraph.
Why this is a layer and not model-level tensors. The five layout tables previously
lived as fields on each layout-aware model, were allocated and randomly initialized, were
counted as parameters, serialized into every checkpoint and updated by the optimizer — and were
never read by any forward pass. Weights that train but do not participate are worse than absent
ones: they consume memory and checkpoint bytes and they make the parameter count describe a
model that does not exist. As a layer the tables sit in Layers, so they are discovered by
TapeTrainingStep.CollectParameters, counted by the parameter automation, and picked up by
BOTH the inference forward and the tape-based training forward with no override on either.
Learned, not sinusoidal, positions. This layer replaces the
EmbeddingLayer + PositionalEncodingLayer pair that used to sit
at the front of these stacks. PositionalEncodingLayer is
SupportsTraining => false — fixed sinusoids — whereas LayoutLM inherits BERT’s LEARNED
position embeddings. The sinusoidal layer was therefore not merely a duplicate of the dead
_positionEmbeddings field; it was the wrong kind of position embedding.
Constructors
| Constructor | Summary |
|---|---|
LayoutEmbeddingLayer(int vocabSize, int hiddenDim, int maxSequenceLength, int maxPosition2D, bool includeTokens) |
Initializes a new LayoutEmbeddingLayer. |
Properties
| Property | Summary |
|---|---|
EmbeddingLayer<T> WordEmbeddings { get; } |
Gets the token embedding table, exposed so a model can load pretrained vectors. |
IReadOnlyList<LayerPort> InputPorts { get; } |
Generated from [TensorPort] declarations. |
IReadOnlyList<LayerPort> OutputPorts { get; } |
Generated from [TensorPort] declarations. |
bool SupportsTraining { get; } |
|
int EmbeddingDim { get; } |
Dimensionality of the per-token vector this layer produces. |
int MaxPosition2D { get; } |
Number of coordinate buckets each 2D table covers. |
int MaxSequenceLength { get; } |
Longest sequence the learned 1D position table can index; later tokens reuse the last row. |
int RowWidth { get; } |
Row width this instance consumes: five with a token column, four in boxes-only mode. |
int VocabularySize { get; } |
Size of the token vocabulary the word table covers. |
Methods
| Method | Summary |
|---|---|
IReadOnlyList<ILayer<T>> GetSubLayers() |
|
IReadOnlyList<OutputAxisContract> OutputAxesFor(int inputRank) |
|
LayerInputDomain GetInputDomain(int[] inputShape) |
Generated from the primary [TensorPort] declaration. |
void ResetState() |
|
void UpdateParameters(T learningRate) |
Fields
| Field | Summary |
|---|---|
int BoxOnlyRowWidth |
Width of a boxes-only row when width and height are derived: (x0, y0, x1, y1). |
int BoxWithExtentRowWidth |
Width of a boxes-only row that carries width and height explicitly: (x0, y0, x1, y1, w, h). |
int PackedRowWidth |
Width of a packed input row: one token ID followed by the four bounding-box coordinates (x0, y0, x1, y1). |