DiT<T>

Models & Types · AiDotNet.Document.LayoutAware

DiT (Document Image Transformer) for document image understanding.

Inheritance: NeuralNetworkBase<T> → DocumentNeuralNetworkBase<T> → DiT<T>

Implements: IDocumentClassifier<T>, ILayoutDetector<T>

For Beginners

DiT learns document understanding from images alone:

  1. Pre-trains on 42 million document images
  2. Uses masked image modeling (predicts missing patches)
  3. Learns document-specific visual patterns

Key features:

  • Pure vision approach (no OCR needed for pre-training)
  • ViT-base/large architectures
  • State-of-the-art on document classification
  • Strong layout analysis performance

Example usage:

How It Works

DiT applies self-supervised pre-training on large-scale document images using a Vision Transformer (ViT) backbone, enabling strong document layout analysis without requiring OCR annotations.

Reference: “DiT: Self-supervised Pre-training for Document Image Transformer” (ACM MM 2022) https://arxiv.org/abs/2203.02378

Constructors

Constructor Summary
DiT(NeuralNetworkArchitecture<T> architecture, DiTOptions options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer, ILossFunction<T> lossFunction) Creates a DiT model using native layers for training and inference.
DiT(NeuralNetworkArchitecture<T> architecture, string onnxModelPath, DiTOptions options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer, ILossFunction<T> lossFunction) Creates a DiT model using a pre-trained ONNX model for inference.

Properties

Property Summary
DiTModelSize ModelSize { get; } Gets the model size variant (base/large).
DocumentType SupportedDocumentTypes { get; }
IReadOnlyList<LayoutElementType> SupportedElementTypes { get; }
IReadOnlyList<string> AvailableCategories { get; }
bool RequiresOCR { get; }
int ExpectedImageSize { get; }
int PatchSize { get; } Gets the patch size for the ViT backbone.

Methods

Method Summary
DocumentClassificationResult<T> ClassifyDocument(Tensor<T> documentImage)
DocumentClassificationResult<T> ClassifyDocument(Tensor<T> documentImage, int topK)
DocumentLayoutResult<T> DetectLayout(Tensor<T> documentImage)
DocumentLayoutResult<T> DetectLayout(Tensor<T> documentImage, double confidenceThreshold)
IEnumerable<DocumentClassificationResult<T>> ClassifyDocumentBatch(IEnumerable<Tensor<T>> documentImages)
ModelMetadata<T> GetModelMetadata()
Tensor<T> EncodeDocument(Tensor<T> documentImage)
string GetModelSummary()
void ValidateInputShape(Tensor<T> documentImage)