Dessurt<T>
Models & Types · AiDotNet.Document.PixelToSequence
Dessurt (Document End-to-end Self-Supervised Understanding and RecogniTion) for document understanding.
Inheritance: NeuralNetworkBase<T> → DocumentNeuralNetworkBase<T> → Dessurt<T>
Implements: IDocumentQA<T>
For Beginners
Dessurt learns document understanding without labels:
- Pre-trains by reconstructing corrupted document images
- Learns to understand text, layout, and visual patterns
- Fine-tunes on downstream tasks with minimal supervision
Key features:
- Self-supervised pre-training (no labels needed)
- Denoising autoencoder objective
- Vision encoder + text decoder architecture
- OCR-free document understanding
Example usage:
How It Works
Dessurt is a self-supervised pre-training approach for document understanding that learns from document images without any labeled data. It uses a denoising autoencoder objective to learn robust document representations.
Reference: “Dessurt: A Dessert for Document Understanding” (arXiv 2022)
https://arxiv.org/abs/2203.16618
Constructors
| Constructor | Summary |
|---|---|
Dessurt(NeuralNetworkArchitecture<T> architecture, DessurtOptions options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer, ILossFunction<T> lossFunction) |
Creates a Dessurt model using native layers for training and inference. |
Dessurt(NeuralNetworkArchitecture<T> architecture, string onnxModelPath, DessurtOptions options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer, ILossFunction<T> lossFunction) |
Creates a Dessurt model using a pre-trained ONNX model for inference. |
Properties
| Property | Summary |
|---|---|
DocumentType SupportedDocumentTypes { get; } |
|
bool RequiresOCR { get; } |
|
int DecoderDim { get; } |
Gets the decoder hidden dimension. |
int EncoderDim { get; } |
Gets the encoder hidden dimension. |
int ExpectedImageSize { get; } |
Methods
| Method | Summary |
|---|---|
Dictionary<string, DocumentQAResult<T>> ExtractFields(Tensor<T> documentImage, IEnumerable<string> fieldPrompts) |
|
DocumentQAResult<T> AnswerQuestion(Tensor<T> documentImage, string question) |
|
DocumentQAResult<T> AnswerQuestion(Tensor<T> documentImage, string question, int maxAnswerLength, double temperature) |
|
IEnumerable<DocumentQAResult<T>> AnswerQuestions(Tensor<T> documentImage, IEnumerable<string> questions) |
|
ModelMetadata<T> GetModelMetadata() |
|
Tensor<T> EncodeDocument(Tensor<T> documentImage) |
|
string GetModelSummary() |
|
void ValidateInputShape(Tensor<T> documentImage) |