RAPIDFlow<T>
Models & Types · AiDotNet.Video.Motion
RAPIDFlow — Recurrent Adaptable Pyramids with Iterative Decoding for efficient optical-flow estimation (Morimitsu 2025, https://arxiv.org/abs/2501.04350).
Inheritance: NeuralNetworkBase<T> → VideoNeuralNetworkBase<T> → OpticalFlowBase<T> → RAPIDFlow<T>
For Beginners
Imagine looking at two video frames side by side and trying to figure out how things moved between them. RAPIDFlow zooms OUT first (the pyramid downsample), figures out the big motions in the blurry low-resolution view (the iterative refinement at the bottom of the pyramid), then progressively zooms IN to sharpen those estimates back to full resolution (the upsample decoder). Working at lower resolutions is what makes it fast — there are far fewer pixels to process at the bottom of the pyramid.
How It Works
Paper-faithful architecture summary:
The previous implementation in this codebase was a flat stack of 10
non-pyramid full-resolution convolutions — that violated both the
“Pyramid” axis of the paper (no multi-scale structure) and the
“Efficient” axis (every conv at full res = ~15× more compute than the
paper-faithful pyramid). Rewriting to the pyramid structure here is
what fixes the generated Training_ShouldReduceLoss /
MoreData_ShouldNotDegrade /
LossStrictlyDecreasesOnMemorizationTask invariants — the
failure mode was real (the model couldn’t be exercised within the
xUnit per-test timeout, not the test budget being too tight).
Example
var rapidFlow = new RAPIDFlow<double>();
var architecture = new NeuralNetworkArchitecture<double>(
inputType: InputType.ThreeDimensional,
taskType: NeuralNetworkTaskType.Regression,
inputHeight: 256, inputWidth: 256, inputDepth: 3, outputSize: 2);
var model = new RAPIDFlow<double>(architecture);
Constructors
| Constructor | Summary |
|---|---|
RAPIDFlow() |
Initializes a new RAPIDFlow with paper-default settings (256×256 RGB inputs, 5 refinement iterations). |
RAPIDFlow(NeuralNetworkArchitecture<T> architecture, RAPIDFlowOptions options) |
Creates a new RAPIDFlow model for native training and inference. |
Methods
| Method | Summary |
|---|---|
ModelMetadata<T> GetModelMetadata() |
|
Tensor<T> EstimateFlow(Tensor<T> frame0, Tensor<T> frame1) |