RAFT<T>
Models & Types · AiDotNet.Video.Motion
Recurrent All-pairs Field Transforms (RAFT) for optical flow estimation.
Inheritance: NeuralNetworkBase<T> → VideoNeuralNetworkBase<T> → OpticalFlowBase<T> → RAFT<T>
For Beginners
RAFT is a state-of-the-art optical flow estimation model that predicts the motion between two consecutive video frames. Optical flow represents how pixels move from one frame to the next, useful for:
- Motion analysis and tracking
- Video stabilization
- Action recognition
- Video compression
- Self-driving car perception
RAFT iteratively refines its flow estimate using a recurrent update mechanism, making it very accurate while remaining efficient.
How It Works
Technical Details:
- Feature extraction using CNN encoder
- 4D correlation volumes for all-pairs matching
- GRU-based iterative update operator
- Multi-scale feature pyramids
Reference: Teed and Deng, “RAFT: Recurrent All-Pairs Field Transforms for Optical Flow” ECCV 2020.
Example
// Create a RAFT model for recurrent optical flow estimation
var raft = new RAFT<double>();
// Or configure with custom architecture
var architecture = new NeuralNetworkArchitecture<double>(
inputType: InputType.ThreeDimensional,
taskType: NeuralNetworkTaskType.Regression,
inputHeight: 256, inputWidth: 256, inputDepth: 3, outputSize: 2);
var model = new RAFT<double>(architecture, options: new RAFTOptions { NumFeatures = 128, NumIterations = 12 });
Constructors
| Constructor | Summary |
|---|---|
RAFT() |
Initializes a new instance with default architecture settings. |
RAFT(NeuralNetworkArchitecture<T> architecture, RAFTOptions options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer) |
Initializes a new instance of the RAFT class. |
Properties
| Property | Summary |
|---|---|
bool SupportsTraining { get; } |
Gets whether training is supported. |
Methods
| Method | Summary |
|---|---|
List<Tensor<T>> EstimateFlowIterative(Tensor<T> frame1, Tensor<T> frame2) |
Estimates optical flow with intermediate flow predictions. |
ModelMetadata<T> GetModelMetadata() |
|
Tensor<T> EstimateFlow(Tensor<T> frame1, Tensor<T> frame2) |
Estimates optical flow between two frames. |
Tensor<T> ForwardForTraining(Tensor<T> input) |