OpticalFlowBase<T>

Base Classes · AiDotNet.Video

Base class for optical flow estimation models that compute dense pixel-wise motion between frames.

Inheritance: NeuralNetworkBase<T> → VideoNeuralNetworkBase<T> → OpticalFlowBase<T>

Implements: IShapeContract

Known derived types: DKM<T>, DPFlow<T>, FlowDiffuser<T>, FlowFormerPlusPlus<T>, FlowFormer<T>, GMFlow<T>, MemFlow<T>, NeuFlowV2<T>, RAFT<T>, RAPIDFlow<T>, RPKNet<T>, RoMa<T>, SEARAFT<T>, SKFlow<T>, UFM<T>, UniMatch<T>, VideoFlow<T>

For Beginners

Optical flow tells you how each pixel moved between two frames. It’s like tracking every single point in the image. The output is a “flow field” where each position stores (dx, dy) - how far that pixel moved horizontally and vertically. This is useful for video stabilization, frame interpolation, action recognition, and more.

How It Works

Optical flow estimation computes per-pixel motion vectors between two consecutive frames. This base class provides:

  • Flow field output (dense 2D displacement vectors)
  • Multi-scale iterative refinement support
  • Forward-backward consistency checking
  • Flow visualization utilities

Derived classes implement specific architectures like RAFT, FlowFormer, SEA-RAFT, etc.

Properties

Property Summary
bool SupportsMultiScale { get; set; } Gets whether this model supports multi-scale processing.
int NumIterations { get; set; } Gets the number of iterative refinement steps.

Methods

Method Summary
IReadOnlyList<OutputAxisContract> OutputAxesFor(int inputRank) The optical-flow family’s law: [Batch, 2, Height, Width].
List<Tensor<T>> EstimateFlowMultiScale(Tensor<T> frame0, Tensor<T> frame1, int numLevels) Estimates optical flow at multiple scales for handling large motions.
T ComputeEndpointError(Tensor<T> estimatedFlow, Tensor<T> groundTruthFlow) Computes the endpoint error (EPE) between estimated and ground truth flow.
Tensor<T> ComputeForwardBackwardConsistency(Tensor<T> forwardFlow, Tensor<T> backwardFlow) Computes forward-backward consistency between two flow fields.
Tensor<T> EstimateFlow(Tensor<T> frame0, Tensor<T> frame1) Estimates optical flow between two frames.