TrOCR<T>
Models & Types · AiDotNet.ComputerVision.OCR.Recognition
TrOCR (Transformer-based OCR) for text recognition.
Inheritance: ModelBase<T, Tensor<T>, Tensor<T>> → VisionTaskModelBase<T> → OCRBase<T> → TrOCR<T>
Implements: IGeneratedParameterRegistrar<T>
For Beginners
TrOCR uses a Vision Transformer (ViT) as the encoder to extract visual features, and a Transformer decoder to generate text autoregressively. This architecture leverages the power of pre-trained language models.
How It Works
Key features:
- Vision Transformer encoder for image understanding
- Transformer decoder with attention for text generation
- Autoregressive decoding with beam search
- Can leverage pre-trained models
Reference: Li et al., “TrOCR: Transformer-based Optical Character Recognition with Pre-trained Models”, AAAI 2023
Constructors
| Constructor | Summary |
|---|---|
TrOCR(OCROptions<T> options) |
Creates a new TrOCR text recognizer. |
Properties
| Property | Summary |
|---|---|
int NumHeads { get; } |
Gets the number of attention heads in the transformer. |
string Name { get; } |
Methods
| Method | Summary |
|---|---|
OCRResult<T> Recognize(Tensor<T> image) |
|
Task LoadWeightsAsync(string pathOrUrl, CancellationToken cancellationToken) |
|
long GetParameterCount() |
|
void SaveWeights(string path) |
|
void Train(Tensor<T> input, Tensor<T> expectedOutput) |
Runs one teacher-forced training step with cross-entropy. |