RAPIDFlow<T>

Models & Types · AiDotNet.Video.Motion

RAPIDFlow — Recurrent Adaptable Pyramids with Iterative Decoding for efficient optical-flow estimation (Morimitsu 2025, https://arxiv.org/abs/2501.04350).

Inheritance: NeuralNetworkBase<T> → VideoNeuralNetworkBase<T> → OpticalFlowBase<T> → RAPIDFlow<T>

For Beginners

Imagine looking at two video frames side by side and trying to figure out how things moved between them. RAPIDFlow zooms OUT first (the pyramid downsample), figures out the big motions in the blurry low-resolution view (the iterative refinement at the bottom of the pyramid), then progressively zooms IN to sharpen those estimates back to full resolution (the upsample decoder). Working at lower resolutions is what makes it fast — there are far fewer pixels to process at the bottom of the pyramid.

How It Works

Paper-faithful architecture summary:

The previous implementation in this codebase was a flat stack of 10 non-pyramid full-resolution convolutions — that violated both the “Pyramid” axis of the paper (no multi-scale structure) and the “Efficient” axis (every conv at full res = ~15× more compute than the paper-faithful pyramid). Rewriting to the pyramid structure here is what fixes the generated Training_ShouldReduceLoss / MoreData_ShouldNotDegrade / LossStrictlyDecreasesOnMemorizationTask invariants — the failure mode was real (the model couldn’t be exercised within the xUnit per-test timeout, not the test budget being too tight).

Example

var rapidFlow = new RAPIDFlow<double>();
            
var architecture = new NeuralNetworkArchitecture<double>(
    inputType: InputType.ThreeDimensional,
    taskType: NeuralNetworkTaskType.Regression,
    inputHeight: 256, inputWidth: 256, inputDepth: 3, outputSize: 2);
var model = new RAPIDFlow<double>(architecture);

Constructors

Constructor Summary
RAPIDFlow() Initializes a new RAPIDFlow with paper-default settings (256×256 RGB inputs, 5 refinement iterations).
RAPIDFlow(NeuralNetworkArchitecture<T> architecture, RAPIDFlowOptions options) Creates a new RAPIDFlow model for native training and inference.

Methods

Method Summary
ModelMetadata<T> GetModelMetadata()
Tensor<T> EstimateFlow(Tensor<T> frame0, Tensor<T> frame1)