LayoutLMv3<T>

Models & Types · AiDotNet.Document.LayoutAware

LayoutLMv3 neural network for document understanding with unified text and image pre-training.

Inheritance: NeuralNetworkBase<T> → DocumentNeuralNetworkBase<T> → LayoutLMv3<T>

Implements: IDocumentQA<T>, ILayoutDetector<T>

For Beginners

LayoutLMv3 understands documents by learning from:

  1. The text content (what the words say)
  2. The visual appearance (what the document looks like)
  3. The layout structure (where elements are positioned)

This makes it excellent for:

  • Extracting information from forms and receipts
  • Understanding document structure
  • Answering questions about document content
  • Classifying document types

Example usage (ONNX mode - for inference with pre-trained models):

Example usage (Native mode - for training):

How It Works

LayoutLMv3 is the third generation of the LayoutLM series from Microsoft Research, featuring unified multimodal pre-training with masked image modeling and masked language modeling on the same architecture.

Reference: “LayoutLMv3: Pre-training for Document AI with Unified Text and Image Masking” https://arxiv.org/abs/2204.08387

Constructors

Constructor Summary
LayoutLMv3(NeuralNetworkArchitecture<T> architecture, LayoutLMv3Options options, ITokenizer tokenizer, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer, ILossFunction<T> lossFunction) Creates a LayoutLMv3 model using native layers for training and inference.
LayoutLMv3(NeuralNetworkArchitecture<T> architecture, string onnxModelPath, ITokenizer tokenizer, LayoutLMv3Options options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer, ILossFunction<T> lossFunction) Creates a LayoutLMv3 model using a pre-trained ONNX model for inference.

Properties

Property Summary
DocumentType SupportedDocumentTypes { get; }
IReadOnlyList<LayoutElementType> SupportedElementTypes { get; }
bool RequiresOCR { get; }
int ExpectedImageSize { get; }

Methods

Method Summary
Dictionary<string, DocumentQAResult<T>> ExtractFields(Tensor<T> documentImage, IEnumerable<string> fieldPrompts)
Dictionary<string, Tensor<T>> GetNamedLayerActivations(Tensor<T> input)
DocumentLayoutResult<T> DetectLayout(Tensor<T> documentImage)
DocumentLayoutResult<T> DetectLayout(Tensor<T> documentImage, double confidenceThreshold)
DocumentQAResult<T> AnswerQuestion(Tensor<T> documentImage, string question)
DocumentQAResult<T> AnswerQuestion(Tensor<T> documentImage, string question, int maxAnswerLength, double temperature)
IEnumerable<DocumentQAResult<T>> AnswerQuestions(Tensor<T> documentImage, IEnumerable<string> questions)
ModelMetadata<T> GetModelMetadata()
Tensor<T> EncodeDocument(Tensor<T> documentImage)
Tensor<T> ForwardForTraining(Tensor<T> input)
string GetModelSummary()
void ValidateInputShape(Tensor<T> documentImage)