MATCHA<T>
Models & Types · AiDotNet.Document.PixelToSequence
MATCHA (Math-Aware Transformer for Chart Harvesting and Analysis) for chart understanding.
Inheritance: NeuralNetworkBase<T> → DocumentNeuralNetworkBase<T> → MATCHA<T>
Implements: IDocumentQA<T>, ITableExtractor<T>
For Beginners
MATCHA specializes in understanding charts:
- Reads bar charts, line graphs, pie charts, scatter plots
- Extracts underlying numerical data
- Answers questions about chart content
- Generates summaries of chart insights
Key features:
- Math-aware pre-training for numerical reasoning
- Pix2Struct-based architecture
- Chart derendering (image to data table)
- Chart QA and summarization
Example usage:
How It Works
MATCHA is designed specifically for understanding charts and plots, combining math-aware pre-training with visual encoding to extract data, answer questions, and summarize chart content.
Reference: “MatCha: Enhancing Visual Language Pretraining with Math Reasoning and Chart Derendering” (ACL 2023)
https://arxiv.org/abs/2212.09662
Constructors
| Constructor | Summary |
|---|---|
MATCHA(NeuralNetworkArchitecture<T> architecture, MATCHAOptions options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer, ILossFunction<T> lossFunction) |
Creates a MATCHA model using native layers for training and inference. |
MATCHA(NeuralNetworkArchitecture<T> architecture, string onnxModelPath, MATCHAOptions options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer, ILossFunction<T> lossFunction) |
Creates a MATCHA model using a pre-trained ONNX model for inference. |
Properties
| Property | Summary |
|---|---|
DocumentType SupportedDocumentTypes { get; } |
|
IReadOnlyList<string> SupportedChartTypes { get; } |
Gets the supported chart types. |
bool RequiresOCR { get; } |
|
bool SupportsBorderedTables { get; } |
|
bool SupportsBorderlessTables { get; } |
|
bool SupportsMergedCells { get; } |
|
int ExpectedImageSize { get; } |
|
int MaxPatchesPerImage { get; } |
Gets the maximum patches per image. |
Methods
| Method | Summary |
|---|---|
Dictionary<string, DocumentQAResult<T>> ExtractFields(Tensor<T> documentImage, IEnumerable<string> fieldPrompts) |
|
Dictionary<string, Tensor<T>> GetNamedLayerActivations(Tensor<T> input) |
|
DocumentQAResult<T> AnswerQuestion(Tensor<T> documentImage, string question) |
|
DocumentQAResult<T> AnswerQuestion(Tensor<T> documentImage, string question, int maxAnswerLength, double temperature) |
|
IEnumerable<DocumentQAResult<T>> AnswerQuestions(Tensor<T> documentImage, IEnumerable<string> questions) |
|
IEnumerable<List<List<string>>> ExtractTableContent(Tensor<T> documentImage) |
|
IEnumerable<TableRegion<T>> DetectTables(Tensor<T> documentImage) |
|
ModelMetadata<T> GetModelMetadata() |
|
TableStructureResult<T> RecognizeStructure(Tensor<T> tableImage) |
|
Tensor<T> EncodeDocument(Tensor<T> documentImage) |
|
Tensor<T> ForwardForTraining(Tensor<T> input) |
|
string ExportTables(Tensor<T> documentImage, TableExportFormat format) |
|
string GetModelSummary() |
|
void ValidateInputShape(Tensor<T> documentImage) |