BezierAlign
Helpers & Utilities · AiDotNet.ComputerVision.OCR.EndToEnd
BezierAlign: rectifies an arbitrarily curved text instance into a rectangular feature map by sampling along its two boundary Bezier curves.
For Beginners
Curved text in a photo is hard to read directly. This straightens it out — it walks along the top and bottom edges of the text and resamples the image into a neat rectangle, so the text recognizer only ever sees flat, horizontal text.
How It Works
This is the second of ABCNet’s two contributions (Liu et al., CVPR 2020, arXiv:2002.10200), and the one that makes the Bezier representation useful rather than merely compact. RoIAlign samples an AXIS-ALIGNED grid and RoIRotate samples a rotated quadrilateral; both are the wrong shape for curved text, so they drag in background and cut off ascenders, and the recognizer then has to cope with a warped, partly-occluded strip. BezierAlign instead samples a grid that FOLLOWS the curve, so the rectified output is upright text regardless of how the instance bends.
The sampling rule, for output pixel (i, j) of an outputHeight x outputWidth map:
t = (j + 0.5) / outputWidth walks along the curve, s = (i + 0.5) / outputHeight walks
across it, and the source location is
(1 - s) * TopCurve(t) + s * BottomCurve(t) — the point s of the way down the segment joining
the two curves at the same t. Half-pixel centres rather than j / outputWidth, matching
RoIAlign’s convention: sampling at pixel corners biases the whole map half a pixel toward the origin.
GRADIENTS REACH THE CONTROL POINTS. Because the Bernstein coefficients depend only on
(i, j), the sampling grid is a LINEAR map of the eight control points, so it is built as a
constant matrix times the control-point tensor through engine ops rather than being precomputed into
a constant grid. A precomputed grid is the obvious simplification and it silently severs the
detection branch: the control points would receive gradient only from their own regression loss and
none from the recognition loss, so the two branches would stop informing each other and the
end-to-end training the paper reports would not be happening.
Methods
| Method | Summary |
|---|---|
Tensor<T> ControlPointTensor(CubicBezierCurve<T> topCurve, CubicBezierCurve<T> bottomCurve) |
Builds the [8, 2] control-point tensor BezierAlign consumes, top curve first. |
Tensor<T> Sample(IEngine engine, Tensor<T> features, Tensor<T> controlPoints, int outputHeight, int outputWidth) |
Rectifies a curved text instance out of a feature map. |
Tensor<T> SamplingCoefficients(int outputHeight, int outputWidth) |
The constant coefficient matrix [outputHeight * outputWidth, 8] mapping control points to sampling locations. |
Fields
| Field | Summary |
|---|---|
int ControlPointCount |
Control points per instance: four along the top edge, then four along the bottom. |