DiT<T>
Models & Types · AiDotNet.Document.LayoutAware
DiT (Document Image Transformer) for document image understanding.
Inheritance: NeuralNetworkBase<T> → DocumentNeuralNetworkBase<T> → DiT<T>
Implements: IDocumentClassifier<T>, ILayoutDetector<T>
For Beginners
DiT learns document understanding from images alone:
- Pre-trains on 42 million document images
- Uses masked image modeling (predicts missing patches)
- Learns document-specific visual patterns
Key features:
- Pure vision approach (no OCR needed for pre-training)
- ViT-base/large architectures
- State-of-the-art on document classification
- Strong layout analysis performance
Example usage:
How It Works
DiT applies self-supervised pre-training on large-scale document images using a Vision Transformer (ViT) backbone, enabling strong document layout analysis without requiring OCR annotations.
Reference: “DiT: Self-supervised Pre-training for Document Image Transformer” (ACM MM 2022)
https://arxiv.org/abs/2203.02378
Constructors
| Constructor | Summary |
|---|---|
DiT(NeuralNetworkArchitecture<T> architecture, DiTOptions options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer, ILossFunction<T> lossFunction) |
Creates a DiT model using native layers for training and inference. |
DiT(NeuralNetworkArchitecture<T> architecture, string onnxModelPath, DiTOptions options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer, ILossFunction<T> lossFunction) |
Creates a DiT model using a pre-trained ONNX model for inference. |
Properties
| Property | Summary |
|---|---|
DiTModelSize ModelSize { get; } |
Gets the model size variant (base/large). |
DocumentType SupportedDocumentTypes { get; } |
|
IReadOnlyList<LayoutElementType> SupportedElementTypes { get; } |
|
IReadOnlyList<string> AvailableCategories { get; } |
|
bool RequiresOCR { get; } |
|
int ExpectedImageSize { get; } |
|
int PatchSize { get; } |
Gets the patch size for the ViT backbone. |
Methods
| Method | Summary |
|---|---|
DocumentClassificationResult<T> ClassifyDocument(Tensor<T> documentImage) |
|
DocumentClassificationResult<T> ClassifyDocument(Tensor<T> documentImage, int topK) |
|
DocumentLayoutResult<T> DetectLayout(Tensor<T> documentImage) |
|
DocumentLayoutResult<T> DetectLayout(Tensor<T> documentImage, double confidenceThreshold) |
|
IEnumerable<DocumentClassificationResult<T>> ClassifyDocumentBatch(IEnumerable<Tensor<T>> documentImages) |
|
ModelMetadata<T> GetModelMetadata() |
|
Tensor<T> EncodeDocument(Tensor<T> documentImage) |
|
string GetModelSummary() |
|
void ValidateInputShape(Tensor<T> documentImage) |