LayoutEmbeddingLayer<T>

Layers · AiDotNet.NeuralNetworks.Layers

Paper-faithful text + 2D-layout embedding front-end for layout-aware document models (Xu et al., KDD 2020, “LayoutLM: Pre-training of Text and Layout for Document Image Understanding”, §3.2 “Model Architecture”).

Inheritance: LayerBase<T> → LayoutEmbeddingLayer<T>

Implements: IShapeContract

For Beginners

A plain language model sees a document as one long line of words, so it cannot tell a table cell from a footnote. This layer hands the model the coordinates of the box each word was printed in, so “500” in the top-right corner of an invoice can be learned to mean something different from “500” in the body text. You give it the word IDs and, optionally, the boxes; it gives back one vector per word that mixes both.

How It Works

LayoutLM’s contribution over BERT is that a token carries WHERE it sits on the page, not just what it says. The paper builds one embedding block that sums a token’s identity, its position in the reading order, and six numbers derived from its bounding box:

The x-axis table is SHARED between the left and right edges, and the y-axis table between the top and bottom edges — the paper uses “two embedding tables to embed x-axis and y-axis features separately”, so a coordinate means the same thing whichever corner it came from. Width and height get their own tables because a box’s SIZE is a different feature from its POSITION: two boxes can start at the same x and be a caption and a paragraph.

Why this is a layer and not model-level tensors. The five layout tables previously lived as fields on each layout-aware model, were allocated and randomly initialized, were counted as parameters, serialized into every checkpoint and updated by the optimizer — and were never read by any forward pass. Weights that train but do not participate are worse than absent ones: they consume memory and checkpoint bytes and they make the parameter count describe a model that does not exist. As a layer the tables sit in Layers, so they are discovered by TapeTrainingStep.CollectParameters, counted by the parameter automation, and picked up by BOTH the inference forward and the tape-based training forward with no override on either.

Learned, not sinusoidal, positions. This layer replaces the EmbeddingLayer + PositionalEncodingLayer pair that used to sit at the front of these stacks. PositionalEncodingLayer is SupportsTraining => false — fixed sinusoids — whereas LayoutLM inherits BERT’s LEARNED position embeddings. The sinusoidal layer was therefore not merely a duplicate of the dead _positionEmbeddings field; it was the wrong kind of position embedding.

Constructors

Constructor Summary
LayoutEmbeddingLayer(int vocabSize, int hiddenDim, int maxSequenceLength, int maxPosition2D, bool includeTokens) Initializes a new LayoutEmbeddingLayer.

Properties

Property Summary
EmbeddingLayer<T> WordEmbeddings { get; } Gets the token embedding table, exposed so a model can load pretrained vectors.
IReadOnlyList<LayerPort> InputPorts { get; } Generated from [TensorPort] declarations.
IReadOnlyList<LayerPort> OutputPorts { get; } Generated from [TensorPort] declarations.
bool SupportsTraining { get; }
int EmbeddingDim { get; } Dimensionality of the per-token vector this layer produces.
int MaxPosition2D { get; } Number of coordinate buckets each 2D table covers.
int MaxSequenceLength { get; } Longest sequence the learned 1D position table can index; later tokens reuse the last row.
int RowWidth { get; } Row width this instance consumes: five with a token column, four in boxes-only mode.
int VocabularySize { get; } Size of the token vocabulary the word table covers.

Methods

Method Summary
IReadOnlyList<ILayer<T>> GetSubLayers()
IReadOnlyList<OutputAxisContract> OutputAxesFor(int inputRank)
LayerInputDomain GetInputDomain(int[] inputShape) Generated from the primary [TensorPort] declaration.
void ResetState()
void UpdateParameters(T learningRate)

Fields

Field Summary
int BoxOnlyRowWidth Width of a boxes-only row when width and height are derived: (x0, y0, x1, y1).
int BoxWithExtentRowWidth Width of a boxes-only row that carries width and height explicitly: (x0, y0, x1, y1, w, h).
int PackedRowWidth Width of a packed input row: one token ID followed by the four bounding-box coordinates (x0, y0, x1, y1).