LayoutXLM<T>
Models & Types · AiDotNet.Document.LayoutAware
LayoutXLM neural network for multilingual document understanding.
Inheritance: NeuralNetworkBase<T> → DocumentNeuralNetworkBase<T> → LayoutXLM<T>
Implements: IDocumentQA<T>, ILayoutDetector<T>
For Beginners
LayoutXLM understands documents in many languages:
- Supports 53 languages out-of-the-box
- Can handle mixed-language documents
- Zero-shot cross-lingual transfer (train on one language, test on another)
Key features:
- XLM-RoBERTa multilingual text encoder
- Visual backbone (ResNeXt-FPN) for image features
- Language-agnostic layout understanding
- Pre-trained on XFUND dataset (7 languages)
Example usage:
How It Works
LayoutXLM extends LayoutLMv2 to support multilingual documents by using XLM-RoBERTa as the text backbone and training on documents from multiple languages.
Reference: “LayoutXLM: Multimodal Pre-training for Multilingual Visually-rich Document Understanding” (ACL 2022)
https://arxiv.org/abs/2104.08836
Constructors
| Constructor | Summary |
|---|---|
LayoutXLM(NeuralNetworkArchitecture<T> architecture, LayoutXLMOptions options, ITokenizer tokenizer, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer, ILossFunction<T> lossFunction) |
Creates a LayoutXLM model using native layers for training and inference. |
LayoutXLM(NeuralNetworkArchitecture<T> architecture, string onnxModelPath, ITokenizer tokenizer, LayoutXLMOptions options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer, ILossFunction<T> lossFunction) |
Creates a LayoutXLM model using a pre-trained ONNX model for inference. |
Properties
| Property | Summary |
|---|---|
DocumentType SupportedDocumentTypes { get; } |
|
IReadOnlyList<LayoutElementType> SupportedElementTypes { get; } |
|
bool RequiresOCR { get; } |
|
int ExpectedImageSize { get; } |
|
int NumLanguages { get; } |
Gets the number of languages supported. |
Methods
| Method | Summary |
|---|---|
Dictionary<string, DocumentQAResult<T>> ExtractFields(Tensor<T> documentImage, IEnumerable<string> fieldPrompts) |
|
Dictionary<string, Tensor<T>> GetNamedLayerActivations(Tensor<T> input) |
|
DocumentLayoutResult<T> DetectLayout(Tensor<T> documentImage) |
|
DocumentLayoutResult<T> DetectLayout(Tensor<T> documentImage, double confidenceThreshold) |
|
DocumentQAResult<T> AnswerQuestion(Tensor<T> documentImage, string question) |
|
DocumentQAResult<T> AnswerQuestion(Tensor<T> documentImage, string question, int maxAnswerLength, double temperature) |
|
IEnumerable<DocumentQAResult<T>> AnswerQuestions(Tensor<T> documentImage, IEnumerable<string> questions) |
|
LayerInputDomain GetInputDomain(int[] inputShape) |
Generated rank-routed external input-domain contract. |
ModelMetadata<T> GetModelMetadata() |
|
Tensor<T> EncodeDocument(Tensor<T> documentImage) |
|
Tensor<T> EncodeMultimodal(Tensor<T> textTokens, Tensor<T> documentImage) |
Full text+image fusion entry (industry-standard LayoutXLM): encodes BOTH a token-ID sequence and a document image through the two visual/text streams and fuses them via the transformer. |
Tensor<T> ForwardForTraining(Tensor<T> input) |
|
string GetModelSummary() |
|
void ValidateInputShape(Tensor<T> documentImage) |