LayoutLMv3<T>
Models & Types · AiDotNet.Document.LayoutAware
LayoutLMv3 neural network for document understanding with unified text and image pre-training.
Inheritance: NeuralNetworkBase<T> → DocumentNeuralNetworkBase<T> → LayoutLMv3<T>
Implements: IDocumentQA<T>, ILayoutDetector<T>
For Beginners
LayoutLMv3 understands documents by learning from:
- The text content (what the words say)
- The visual appearance (what the document looks like)
- The layout structure (where elements are positioned)
This makes it excellent for:
- Extracting information from forms and receipts
- Understanding document structure
- Answering questions about document content
- Classifying document types
Example usage (ONNX mode - for inference with pre-trained models):
Example usage (Native mode - for training):
How It Works
LayoutLMv3 is the third generation of the LayoutLM series from Microsoft Research, featuring unified multimodal pre-training with masked image modeling and masked language modeling on the same architecture.
Reference: “LayoutLMv3: Pre-training for Document AI with Unified Text and Image Masking”
https://arxiv.org/abs/2204.08387
Constructors
| Constructor | Summary |
|---|---|
LayoutLMv3(NeuralNetworkArchitecture<T> architecture, LayoutLMv3Options options, ITokenizer tokenizer, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer, ILossFunction<T> lossFunction) |
Creates a LayoutLMv3 model using native layers for training and inference. |
LayoutLMv3(NeuralNetworkArchitecture<T> architecture, string onnxModelPath, ITokenizer tokenizer, LayoutLMv3Options options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer, ILossFunction<T> lossFunction) |
Creates a LayoutLMv3 model using a pre-trained ONNX model for inference. |
Properties
| Property | Summary |
|---|---|
DocumentType SupportedDocumentTypes { get; } |
|
IReadOnlyList<LayoutElementType> SupportedElementTypes { get; } |
|
bool RequiresOCR { get; } |
|
int ExpectedImageSize { get; } |
Methods
| Method | Summary |
|---|---|
Dictionary<string, DocumentQAResult<T>> ExtractFields(Tensor<T> documentImage, IEnumerable<string> fieldPrompts) |
|
Dictionary<string, Tensor<T>> GetNamedLayerActivations(Tensor<T> input) |
|
DocumentLayoutResult<T> DetectLayout(Tensor<T> documentImage) |
|
DocumentLayoutResult<T> DetectLayout(Tensor<T> documentImage, double confidenceThreshold) |
|
DocumentQAResult<T> AnswerQuestion(Tensor<T> documentImage, string question) |
|
DocumentQAResult<T> AnswerQuestion(Tensor<T> documentImage, string question, int maxAnswerLength, double temperature) |
|
IEnumerable<DocumentQAResult<T>> AnswerQuestions(Tensor<T> documentImage, IEnumerable<string> questions) |
|
ModelMetadata<T> GetModelMetadata() |
|
Tensor<T> EncodeDocument(Tensor<T> documentImage) |
|
Tensor<T> ForwardForTraining(Tensor<T> input) |
|
string GetModelSummary() |
|
void ValidateInputShape(Tensor<T> documentImage) |