LiLT<T>
Models & Types · AiDotNet.Document.LayoutAware
LiLT (Language-Independent Layout Transformer) for document understanding.
Inheritance: NeuralNetworkBase<T> → DocumentNeuralNetworkBase<T> → LiLT<T>
Implements: IDocumentQA<T>, ILayoutDetector<T>
For Beginners
LiLT is designed for maximum flexibility:
- Layout understanding is learned separately from text
- Can plug in ANY language model (BERT, RoBERTa, XLM-R, etc.)
- Supports any language without retraining the layout part
Key features:
- BiACM (Bi-directional Attention Complementation Mechanism)
- Separate text and layout streams
- Works with any pre-trained text encoder
- Language-agnostic layout understanding
Example usage:
How It Works
LiLT separates the text and layout modalities during pre-training, enabling the layout model to be combined with ANY pre-trained text model at fine-tuning time, providing true language independence.
Reference: “LiLT: A Simple yet Effective Language-Independent Layout Transformer” (ACL 2022)
https://arxiv.org/abs/2202.13669
Constructors
| Constructor | Summary |
|---|---|
LiLT(NeuralNetworkArchitecture<T> architecture, LiLTOptions options, ITokenizer tokenizer, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer, ILossFunction<T> lossFunction) |
Creates a LiLT model using native layers for training and inference. |
LiLT(NeuralNetworkArchitecture<T> architecture, string onnxModelPath, ITokenizer tokenizer, LiLTOptions options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer, ILossFunction<T> lossFunction) |
Creates a LiLT model using a pre-trained ONNX model for inference. |
Properties
| Property | Summary |
|---|---|
DocumentType SupportedDocumentTypes { get; } |
|
IReadOnlyList<LayoutElementType> SupportedElementTypes { get; } |
|
bool RequiresOCR { get; } |
|
int ExpectedImageSize { get; } |
|
string TextBackbone { get; } |
Gets the text backbone model name. |
Methods
| Method | Summary |
|---|---|
Dictionary<string, DocumentQAResult<T>> ExtractFields(Tensor<T> documentImage, IEnumerable<string> fieldPrompts) |
|
Dictionary<string, Tensor<T>> GetNamedLayerActivations(Tensor<T> input) |
Sends an input through the BiACM dual stream, unpacking bounding boxes when they are present. |
DocumentLayoutResult<T> DetectLayout(Tensor<T> documentImage) |
|
DocumentLayoutResult<T> DetectLayout(Tensor<T> documentImage, double confidenceThreshold) |
|
DocumentQAResult<T> AnswerQuestion(Tensor<T> documentImage, string question) |
|
DocumentQAResult<T> AnswerQuestion(Tensor<T> documentImage, string question, int maxAnswerLength, double temperature) |
|
IEnumerable<DocumentQAResult<T>> AnswerQuestions(Tensor<T> documentImage, IEnumerable<string> questions) |
|
ModelMetadata<T> GetModelMetadata() |
|
Tensor<T> EncodeDocument(Tensor<T> documentImage) |
|
Tensor<T> EncodeDualStream(Tensor<T> textTokens, Tensor<T> layoutBoxes) |
Full dual-stream fusion entry (industry-standard LiLT): encodes text token IDs AND their layout bounding boxes through the coupled BiACM transformer. |
Tensor<T> ForwardForTraining(Tensor<T> input) |
|
string GetModelSummary() |
|
void ValidateInputShape(Tensor<T> documentImage) |