LayoutLMv2<T>

Models & Types · AiDotNet.Document.LayoutAware

LayoutLMv2 neural network for document understanding with visual features.

Inheritance: NeuralNetworkBase<T> → DocumentNeuralNetworkBase<T> → LayoutLMv2<T>

Implements: IDocumentQA<T>, ILayoutDetector<T>

For Beginners

LayoutLMv2 improves on v1 by also looking at the actual image:

  1. Text content (what the words say)
  2. Layout structure (where words are positioned)
  3. Visual appearance (what the document looks like)

Key improvements over v1:

  • Visual backbone (ResNeXt-FPN) for image features
  • Spatial-aware self-attention mechanism
  • Pre-training on both text-layout and image-text-layout alignment

Example usage:

How It Works

LayoutLMv2 extends LayoutLM by adding visual features from a CNN backbone, enabling the model to understand documents through text, layout, AND image features.

Reference: “LayoutLMv2: Multi-modal Pre-training for Visually-rich Document Understanding” (ACL 2021) https://arxiv.org/abs/2012.14740

Constructors

Constructor Summary
LayoutLMv2(NeuralNetworkArchitecture<T> architecture, LayoutLMv2Options options, ITokenizer tokenizer, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer, ILossFunction<T> lossFunction) Creates a LayoutLMv2 model using native layers for training and inference.
LayoutLMv2(NeuralNetworkArchitecture<T> architecture, string onnxModelPath, ITokenizer tokenizer, LayoutLMv2Options options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer, ILossFunction<T> lossFunction) Creates a LayoutLMv2 model using a pre-trained ONNX model for inference.

Properties

Property Summary
DocumentType SupportedDocumentTypes { get; }
IReadOnlyList<LayoutElementType> SupportedElementTypes { get; }
bool RequiresOCR { get; }
int ExpectedImageSize { get; }

Methods

Method Summary
Dictionary<string, DocumentQAResult<T>> ExtractFields(Tensor<T> documentImage, IEnumerable<string> fieldPrompts)
Dictionary<string, Tensor<T>> GetNamedLayerActivations(Tensor<T> input)
DocumentLayoutResult<T> DetectLayout(Tensor<T> documentImage)
DocumentLayoutResult<T> DetectLayout(Tensor<T> documentImage, double confidenceThreshold)
DocumentQAResult<T> AnswerQuestion(Tensor<T> documentImage, string question)
DocumentQAResult<T> AnswerQuestion(Tensor<T> documentImage, string question, int maxAnswerLength, double temperature)
IEnumerable<DocumentQAResult<T>> AnswerQuestions(Tensor<T> documentImage, IEnumerable<string> questions)
LayerInputDomain GetInputDomain(int[] inputShape) Generated rank-routed external input-domain contract.
ModelMetadata<T> GetModelMetadata()
Tensor<T> EncodeDocument(Tensor<T> documentImage)
Tensor<T> EncodeMultimodal(Tensor<T> textTokens, Tensor<T> documentImage) Full text+image fusion entry (industry-standard LayoutLMv2): encodes BOTH a token-ID sequence and a document image and fuses them through the multimodal transformer.
Tensor<T> ForwardForTraining(Tensor<T> input)
string GetModelSummary()
void ValidateInputShape(Tensor<T> documentImage)