Nougat<T>
Models & Types · AiDotNet.Document.PixelToSequence
Nougat neural network for academic document understanding.
Inheritance: NeuralNetworkBase<T> → DocumentNeuralNetworkBase<T> → Nougat<T>
Implements: IDocumentQA<T>
For Beginners
Nougat excels at:
- Converting PDF pages to Markdown
- Preserving mathematical equations (LaTeX)
- Understanding document structure (sections, tables, figures)
- Processing scientific papers without OCR
Example usage:
How It Works
Nougat (Neural Optical Understanding for Academic Documents) is an OCR-free transformer model specifically designed to parse academic papers from PDF images into structured Markdown format with mathematical notation support.
Reference: “Nougat: Neural Optical Understanding for Academic Documents” (arXiv 2023)
https://arxiv.org/abs/2308.13418
Constructors
| Constructor | Summary |
|---|---|
Nougat(NeuralNetworkArchitecture<T> architecture, NougatOptions options, ITokenizer tokenizer, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer, ILossFunction<T> lossFunction) |
Creates a Nougat model using native layers for training and inference. |
Nougat(NeuralNetworkArchitecture<T> architecture, string onnxModelPath, ITokenizer tokenizer, NougatOptions options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer, ILossFunction<T> lossFunction) |
Creates a Nougat model using a pre-trained ONNX model for inference. |
Properties
| Property | Summary |
|---|---|
DocumentType SupportedDocumentTypes { get; } |
|
bool RequiresOCR { get; } |
|
bool SupportsLatex { get; } |
Gets whether this model supports LaTeX equation output. |
int ExpectedImageSize { get; } |
Methods
| Method | Summary |
|---|---|
Dictionary<string, DocumentQAResult<T>> ExtractFields(Tensor<T> documentImage, IEnumerable<string> fieldPrompts) |
|
Dictionary<string, Tensor<T>> GetNamedLayerActivations(Tensor<T> input) |
|
DocumentQAResult<T> AnswerQuestion(Tensor<T> documentImage, string question) |
|
DocumentQAResult<T> AnswerQuestion(Tensor<T> documentImage, string question, int maxAnswerLength, double temperature) |
|
IEnumerable<DocumentQAResult<T>> AnswerQuestions(Tensor<T> documentImage, IEnumerable<string> questions) |
|
IEnumerable<string> ExtractEquations(Tensor<T> pageImage) |
Parses a PDF page and extracts equations in LaTeX format. |
ModelMetadata<T> GetModelMetadata() |
|
Tensor<T> EncodeDocument(Tensor<T> documentImage) |
|
string GetModelSummary() |
|
string ParseAcademicDocument(Tensor<T> pageImage) |
Parses an academic document page and generates Markdown output. |
void ValidateInputShape(Tensor<T> documentImage) |