DocPedia<T>
Models & Types · AiDotNet.VisionLanguage.Document
DocPedia: frequency-domain document understanding model.
Inheritance: NeuralNetworkBase<T> → VisionLanguageModelBase<T> → DocPedia<T>
Implements: IDocumentUnderstandingModel<T>
For Beginners
DocPedia is a document understanding model that processes documents in the frequency domain for text and layout analysis. Default values follow the original paper settings.
How It Works
DocPedia (2024) unleashes the power of large multimodal models for document understanding by operating in the frequency domain. It converts document images into frequency representations via DCT transforms, enabling the model to capture both structural layout patterns and fine text details without requiring explicit OCR preprocessing.
References:
- Paper: “DocPedia: Unleashing the Power of Large Multimodal Model in the Frequency Domain” (2024)
Example
// Create a DocPedia model for frequency-domain document understanding
// with DCT-based visual encoding for text and layout analysis
var architecture = new NeuralNetworkArchitecture<double>(
inputType: InputType.TwoDimensional,
taskType: NeuralNetworkTaskType.ImageClassification,
inputHeight: 224, inputWidth: 224, inputDepth: 3, outputSize: 512);
// ONNX inference mode with pre-trained model
var model = new DocPedia<double>(architecture, "docpedia.onnx");
// Training mode with native layers
var trainModel = new DocPedia<double>(architecture, new DocPediaOptions());
Constructors
| Constructor | Summary |
|---|---|
DocPedia(NeuralNetworkArchitecture<T> architecture, DocPediaOptions options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer) |
|
DocPedia(NeuralNetworkArchitecture<T> architecture, string modelPath, DocPediaOptions options) |
Properties
| Property | Summary |
|---|---|
bool IsOcrFree { get; } |
|
int DecoderEmbeddingDim { get; } |
|
int EmbeddingDimension { get; } |
|
int MaxGenerationLength { get; } |
Methods
| Method | Summary |
|---|---|
ModelMetadata<T> GetModelMetadata() |
|
Tensor<T> AnswerDocumentQuestion(Tensor<T> documentImage, string question) |
|
Tensor<T> EncodeImage(Tensor<T> image) |
|
Tensor<T> ExtractText(Tensor<T> documentImage) |
|
Tensor<T> GenerateFromImage(Tensor<T> image, string prompt) |
Generates text from a document image using DocPedia’s frequency-domain pipeline. |