BezierAlign

Helpers & Utilities · AiDotNet.ComputerVision.OCR.EndToEnd

BezierAlign: rectifies an arbitrarily curved text instance into a rectangular feature map by sampling along its two boundary Bezier curves.

For Beginners

Curved text in a photo is hard to read directly. This straightens it out — it walks along the top and bottom edges of the text and resamples the image into a neat rectangle, so the text recognizer only ever sees flat, horizontal text.

How It Works

This is the second of ABCNet’s two contributions (Liu et al., CVPR 2020, arXiv:2002.10200), and the one that makes the Bezier representation useful rather than merely compact. RoIAlign samples an AXIS-ALIGNED grid and RoIRotate samples a rotated quadrilateral; both are the wrong shape for curved text, so they drag in background and cut off ascenders, and the recognizer then has to cope with a warped, partly-occluded strip. BezierAlign instead samples a grid that FOLLOWS the curve, so the rectified output is upright text regardless of how the instance bends.

The sampling rule, for output pixel (i, j) of an outputHeight x outputWidth map: t = (j + 0.5) / outputWidth walks along the curve, s = (i + 0.5) / outputHeight walks across it, and the source location is (1 - s) * TopCurve(t) + s * BottomCurve(t) — the point s of the way down the segment joining the two curves at the same t. Half-pixel centres rather than j / outputWidth, matching RoIAlign’s convention: sampling at pixel corners biases the whole map half a pixel toward the origin.

GRADIENTS REACH THE CONTROL POINTS. Because the Bernstein coefficients depend only on (i, j), the sampling grid is a LINEAR map of the eight control points, so it is built as a constant matrix times the control-point tensor through engine ops rather than being precomputed into a constant grid. A precomputed grid is the obvious simplification and it silently severs the detection branch: the control points would receive gradient only from their own regression loss and none from the recognition loss, so the two branches would stop informing each other and the end-to-end training the paper reports would not be happening.

Methods

Method Summary
Tensor<T> ControlPointTensor(CubicBezierCurve<T> topCurve, CubicBezierCurve<T> bottomCurve) Builds the [8, 2] control-point tensor BezierAlign consumes, top curve first.
Tensor<T> Sample(IEngine engine, Tensor<T> features, Tensor<T> controlPoints, int outputHeight, int outputWidth) Rectifies a curved text instance out of a feature map.
Tensor<T> SamplingCoefficients(int outputHeight, int outputWidth) The constant coefficient matrix [outputHeight * outputWidth, 8] mapping control points to sampling locations.

Fields

Field Summary
int ControlPointCount Control points per instance: four along the top edge, then four along the bottom.