Dessurt<T>

Models & Types · AiDotNet.Document.PixelToSequence

Dessurt (Document End-to-end Self-Supervised Understanding and RecogniTion) for document understanding.

Inheritance: NeuralNetworkBase<T> → DocumentNeuralNetworkBase<T> → Dessurt<T>

Implements: IDocumentQA<T>

For Beginners

Dessurt learns document understanding without labels:

  1. Pre-trains by reconstructing corrupted document images
  2. Learns to understand text, layout, and visual patterns
  3. Fine-tunes on downstream tasks with minimal supervision

Key features:

  • Self-supervised pre-training (no labels needed)
  • Denoising autoencoder objective
  • Vision encoder + text decoder architecture
  • OCR-free document understanding

Example usage:

How It Works

Dessurt is a self-supervised pre-training approach for document understanding that learns from document images without any labeled data. It uses a denoising autoencoder objective to learn robust document representations.

Reference: “Dessurt: A Dessert for Document Understanding” (arXiv 2022) https://arxiv.org/abs/2203.16618

Constructors

Constructor Summary
Dessurt(NeuralNetworkArchitecture<T> architecture, DessurtOptions options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer, ILossFunction<T> lossFunction) Creates a Dessurt model using native layers for training and inference.
Dessurt(NeuralNetworkArchitecture<T> architecture, string onnxModelPath, DessurtOptions options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer, ILossFunction<T> lossFunction) Creates a Dessurt model using a pre-trained ONNX model for inference.

Properties

Property Summary
DocumentType SupportedDocumentTypes { get; }
bool RequiresOCR { get; }
int DecoderDim { get; } Gets the decoder hidden dimension.
int EncoderDim { get; } Gets the encoder hidden dimension.
int ExpectedImageSize { get; }

Methods

Method Summary
Dictionary<string, DocumentQAResult<T>> ExtractFields(Tensor<T> documentImage, IEnumerable<string> fieldPrompts)
DocumentQAResult<T> AnswerQuestion(Tensor<T> documentImage, string question)
DocumentQAResult<T> AnswerQuestion(Tensor<T> documentImage, string question, int maxAnswerLength, double temperature)
IEnumerable<DocumentQAResult<T>> AnswerQuestions(Tensor<T> documentImage, IEnumerable<string> questions)
ModelMetadata<T> GetModelMetadata()
Tensor<T> EncodeDocument(Tensor<T> documentImage)
string GetModelSummary()
void ValidateInputShape(Tensor<T> documentImage)