OpticalFlowBase<T>
Base Classes · AiDotNet.Video
Base class for optical flow estimation models that compute dense pixel-wise motion between frames.
Inheritance: NeuralNetworkBase<T> → VideoNeuralNetworkBase<T> → OpticalFlowBase<T>
Implements: IShapeContract
Known derived types: DKM<T>, DPFlow<T>, FlowDiffuser<T>, FlowFormerPlusPlus<T>, FlowFormer<T>, GMFlow<T>, MemFlow<T>, NeuFlowV2<T>, RAFT<T>, RAPIDFlow<T>, RPKNet<T>, RoMa<T>, SEARAFT<T>, SKFlow<T>, UFM<T>, UniMatch<T>, VideoFlow<T>
For Beginners
Optical flow tells you how each pixel moved between two frames. It’s like tracking every single point in the image. The output is a “flow field” where each position stores (dx, dy) - how far that pixel moved horizontally and vertically. This is useful for video stabilization, frame interpolation, action recognition, and more.
How It Works
Optical flow estimation computes per-pixel motion vectors between two consecutive frames. This base class provides:
- Flow field output (dense 2D displacement vectors)
- Multi-scale iterative refinement support
- Forward-backward consistency checking
- Flow visualization utilities
Derived classes implement specific architectures like RAFT, FlowFormer, SEA-RAFT, etc.
Properties
| Property | Summary |
|---|---|
bool SupportsMultiScale { get; set; } |
Gets whether this model supports multi-scale processing. |
int NumIterations { get; set; } |
Gets the number of iterative refinement steps. |
Methods
| Method | Summary |
|---|---|
IReadOnlyList<OutputAxisContract> OutputAxesFor(int inputRank) |
The optical-flow family’s law: [Batch, 2, Height, Width]. |
List<Tensor<T>> EstimateFlowMultiScale(Tensor<T> frame0, Tensor<T> frame1, int numLevels) |
Estimates optical flow at multiple scales for handling large motions. |
T ComputeEndpointError(Tensor<T> estimatedFlow, Tensor<T> groundTruthFlow) |
Computes the endpoint error (EPE) between estimated and ground truth flow. |
Tensor<T> ComputeForwardBackwardConsistency(Tensor<T> forwardFlow, Tensor<T> backwardFlow) |
Computes forward-backward consistency between two flow fields. |
Tensor<T> EstimateFlow(Tensor<T> frame0, Tensor<T> frame1) |
Estimates optical flow between two frames. |