InfographicVQA<T>
Models & Types · AiDotNet.Document.VisionLanguage
InfographicVQA for visual question answering on infographics.
Inheritance: NeuralNetworkBase<T> → DocumentNeuralNetworkBase<T> → InfographicVQA<T>
Implements: IDocumentQA<T>
For Beginners
InfographicVQA specializes in complex visual documents:
- Understands mixed content (text, charts, icons, diagrams)
- Handles complex multi-column layouts
- Performs visual reasoning across different element types
- Extracts information from visually rich documents
Key features:
- Multi-scale visual processing
- OCR integration for text extraction
- Visual reasoning across elements
- Trained on InfographicsVQA dataset
Example usage:
How It Works
InfographicVQA is designed to understand and answer questions about infographics, which combine text, icons, charts, diagrams, and other visual elements in complex layouts.
Reference: “InfographicVQA” (WACV 2022)
https://arxiv.org/abs/2104.12756
Constructors
| Constructor | Summary |
|---|---|
InfographicVQA(NeuralNetworkArchitecture<T> architecture, InfographicVQAOptions options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer, ILossFunction<T> lossFunction) |
Creates an InfographicVQA model using native layers for training and inference. |
InfographicVQA(NeuralNetworkArchitecture<T> architecture, string onnxModelPath, InfographicVQAOptions options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer, ILossFunction<T> lossFunction) |
Creates an InfographicVQA model using a pre-trained ONNX model for inference. |
Properties
| Property | Summary |
|---|---|
DocumentType SupportedDocumentTypes { get; } |
|
IReadOnlyList<string> SupportedElementTypes { get; } |
Gets the supported infographic element types. |
bool RequiresOCR { get; } |
|
int ExpectedImageSize { get; } |
|
int FusionDim { get; } |
Gets the fusion dimension. |
int VisionDim { get; } |
Gets the vision encoder dimension. |
Methods
| Method | Summary |
|---|---|
Dictionary<string, DocumentQAResult<T>> ExtractFields(Tensor<T> documentImage, IEnumerable<string> fieldPrompts) |
|
Dictionary<string, Tensor<T>> GetNamedLayerActivations(Tensor<T> input) |
|
DocumentQAResult<T> AnswerQuestion(Tensor<T> documentImage, string question) |
|
DocumentQAResult<T> AnswerQuestion(Tensor<T> documentImage, string question, int maxAnswerLength, double temperature) |
|
IEnumerable<DocumentQAResult<T>> AnswerQuestions(Tensor<T> documentImage, IEnumerable<string> questions) |
|
ModelMetadata<T> GetModelMetadata() |
|
Tensor<T> EncodeDocument(Tensor<T> documentImage) |
|
Tensor<T> ForwardForTraining(Tensor<T> input) |
|
string GetModelSummary() |
|
void ValidateInputShape(Tensor<T> documentImage) |