DocFormer<T>
Models & Types · AiDotNet.Document.LayoutAware
DocFormer neural network for end-to-end document understanding.
Inheritance: NeuralNetworkBase<T> → DocumentNeuralNetworkBase<T> → DocFormer<T>
Implements: IDocumentClassifier<T>, ILayoutDetector<T>
For Beginners
DocFormer combines three types of information:
- Text content (what the words say)
- Visual features (what the document looks like)
- Spatial layout (where elements are positioned)
Unlike LayoutLM which adds position embeddings to text, DocFormer uses shared spatial encodings that align all three modalities in the same coordinate space.
Example usage:
How It Works
DocFormer is a multi-modal transformer that jointly learns text, visual, and spatial features for document understanding tasks. It uses shared spatial encodings across all modalities.
Reference: “DocFormer: End-to-End Transformer for Document Understanding” (ICCV 2021)
https://arxiv.org/abs/2106.11539
Constructors
| Constructor | Summary |
|---|---|
DocFormer(NeuralNetworkArchitecture<T> architecture, DocFormerOptions options, ITokenizer tokenizer, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer, ILossFunction<T> lossFunction) |
Creates a DocFormer model using native layers for training and inference. |
DocFormer(NeuralNetworkArchitecture<T> architecture, string onnxModelPath, ITokenizer tokenizer, DocFormerOptions options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer, ILossFunction<T> lossFunction) |
Creates a DocFormer model using a pre-trained ONNX model for inference. |
Properties
| Property | Summary |
|---|---|
DocumentType SupportedDocumentTypes { get; } |
|
IReadOnlyList<LayoutElementType> SupportedElementTypes { get; } |
|
IReadOnlyList<string> AvailableCategories { get; } |
Gets the available document classification categories. |
bool RequiresOCR { get; } |
|
int ExpectedImageSize { get; } |
Methods
| Method | Summary |
|---|---|
Dictionary<string, Tensor<T>> GetNamedLayerActivations(Tensor<T> input) |
|
DocumentClassificationResult<T> ClassifyDocument(Tensor<T> documentImage) |
|
DocumentClassificationResult<T> ClassifyDocument(Tensor<T> documentImage, int topK) |
|
DocumentLayoutResult<T> DetectLayout(Tensor<T> documentImage) |
|
DocumentLayoutResult<T> DetectLayout(Tensor<T> documentImage, double confidenceThreshold) |
|
LayerInputDomain GetInputDomain(int[] inputShape) |
|
ModelMetadata<T> GetModelMetadata() |
|
Tensor<T> EncodeDocument(Tensor<T> documentImage) |
|
Tensor<T> ForwardForTraining(Tensor<T> input) |
|
Tensor<T> PredictDocument(Tensor<T> packedTokens, Tensor<T> pageImage) |
Token logits [S, numClasses] for packed rows [S, 5] (token id, x0, y0, x1, y1) and the page image [3, H, W], which drives the ResNet-50 visual stream. |
string GetModelSummary() |
|
void ValidateInputShape(Tensor<T> documentImage) |