ABCNet<T>
Models & Types · AiDotNet.ComputerVision.OCR.EndToEnd
ABCNet: single-shot scene text spotting that regresses a cubic Bezier boundary per text instance and rectifies each instance with BezierAlign before recognizing it.
Inheritance: NeuralNetworkBase<T> → ABCNet<T>
Implements: ICompositeLoss<T>
For Beginners
Text in real photos is often curved — around a logo, along a sign. ABCNet describes each piece of text with a smooth curve along its top and bottom edges, uses those curves to straighten the text out, and then reads the straightened version. It finds and reads text in one pass rather than two separate stages.
How It Works
Liu, Chen, Shen, He, Jin and Wang, “ABCNet: Real-time Scene Text Spotting with Adaptive Bezier-Curve Network” (CVPR 2020 oral, pp. 9809-9818, arXiv:2002.10200).
The two contributions live in CubicBezierCurve and BezierAlign, each
independently testable; this class is what composes them into a spotter. What makes it ABCNet rather
than a generic two-stage pipeline is that the SAME feature map feeds both branches and the curve
parameters are what connect them: the detection head regresses eight control points, and BezierAlign
reads the recognition branch’s input through those very control points. A pipeline that detects a box,
crops the image, and hands the crop to a separate recognizer — which is what
SceneTextReader does, honestly and usefully — shares no gradient between the two and
cannot represent a curved instance at all.
SINGLE SHOT, and that is the source of the paper’s real-time claim: text/not-text scores and the 16 curve coordinates come from two 1x1 convolutions over one shared feature map, with no proposal stage and no per-anchor refinement.
The recognition branch is deliberately lightweight — convolutions plus a per-column classifier over the rectified strip, trained with CTC. The paper’s argument is that BezierAlign has already done the hard part: once the instance is straightened, an attention-based recognizer earns very little over a cheap one, and its cost would remove the speed advantage.
Example
var image = Tensor<double>.CreateRandom(1, 3, 32, 32);
var model = new ABCNet<double>(new ABCNetOptions<double> { InputHeight = 256, InputWidth = 256 });
foreach (var instance in model.Spot(image))
Console.WriteLine($"score {instance.Score}, {instance.CharacterIndices.Count} characters");
Constructors
| Constructor | Summary |
|---|---|
ABCNet() |
Creates the model with the paper’s defaults. |
ABCNet(ABCNetOptions<T> options, NeuralNetworkArchitecture<T> architecture, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer, ILossFunction<T> lossFunction) |
Creates the model. |
Properties
| Property | Summary |
|---|---|
IReadOnlyList<OutputSpec<T>> DeclaredOutputs { get; } |
|
int FeatureHeight { get; } |
Gets the detection feature-map height. |
int FeatureWidth { get; } |
Gets the detection feature-map width. |
Methods
| Method | Summary |
|---|---|
IReadOnlyList<Tensor<T>> ComputeOutputs(Tensor<T> input) |
|
IReadOnlyList<TextInstance<T>> DetectInstances(Tensor<T> image) |
Detects text instances, decoding each feature position’s regressed offsets into control points. |
IReadOnlyList<TextInstance<T>> Spot(Tensor<T> image) |
Detects and recognizes every text instance in one pass over the shared features. |
IReadOnlyList<int> CtcGreedyDecode(Tensor<T> logits) |
Greedy CTC decode: argmax per column, then collapse repeats and drop blanks. |
ModelMetadata<T> GetModelMetadata() |
|
Tensor<T> ForwardForTraining(Tensor<T> input) |
|
Tensor<T> RecognizeRectified(Tensor<T> features, Tensor<T> controlPoints) |
Rectifies one instance with BezierAlign and returns per-column character logits. |
Fields
| Field | Summary |
|---|---|
int BezierCoordinateCount |
Coordinates the Bezier head regresses: 8 control points, (x, y) each. |
int ExpectedLayerCount |
Total layers this model routes, in order: backbone, then heads, then recognition. |