TrOCR<T>

Models & Types · AiDotNet.ComputerVision.OCR.Recognition

TrOCR (Transformer-based OCR) for text recognition.

Inheritance: ModelBase<T, Tensor<T>, Tensor<T>> → VisionTaskModelBase<T> → OCRBase<T> → TrOCR<T>

Implements: IGeneratedParameterRegistrar<T>

For Beginners

TrOCR uses a Vision Transformer (ViT) as the encoder to extract visual features, and a Transformer decoder to generate text autoregressively. This architecture leverages the power of pre-trained language models.

How It Works

Key features:

  • Vision Transformer encoder for image understanding
  • Transformer decoder with attention for text generation
  • Autoregressive decoding with beam search
  • Can leverage pre-trained models

Reference: Li et al., “TrOCR: Transformer-based Optical Character Recognition with Pre-trained Models”, AAAI 2023

Constructors

Constructor Summary
TrOCR(OCROptions<T> options) Creates a new TrOCR text recognizer.

Properties

Property Summary
int NumHeads { get; } Gets the number of attention heads in the transformer.
string Name { get; }

Methods

Method Summary
OCRResult<T> Recognize(Tensor<T> image)
Task LoadWeightsAsync(string pathOrUrl, CancellationToken cancellationToken)
long GetParameterCount()
void SaveWeights(string path)
void Train(Tensor<T> input, Tensor<T> expectedOutput) Runs one teacher-forced training step with cross-entropy.