RIFE<T>

Models & Types · AiDotNet.Video.FrameInterpolation

Real-time Intermediate Flow Estimation (RIFE) for video frame interpolation.

Inheritance: NeuralNetworkBase<T> → VideoNeuralNetworkBase<T> → FrameInterpolationBase<T> → RIFE<T>

For Beginners

RIFE is a state-of-the-art model for generating intermediate frames between two existing video frames. This is useful for:

  • Increasing video frame rate (e.g., 24fps to 60fps)
  • Creating slow-motion effects
  • Smoothing video playback
  • Reducing temporal aliasing

RIFE uses a privileged distillation approach with intermediate flow estimation to create realistic frames at arbitrary positions between input frames.

How It Works

Technical Details:

  • Uses IFNet architecture for intermediate flow estimation
  • Coarse-to-fine flow refinement across multiple scales
  • Context-aware fusion with feature maps
  • Supports arbitrary timestep interpolation (not just midpoint)

Reference: Huang et al., “RIFE: Real-Time Intermediate Flow Estimation for Video Frame Interpolation” ECCV 2022.

Example

// Create a RIFE model for real-time video frame interpolation
var rife = new RIFE<double>();
            
// Or configure with custom flow estimation parameters. inputDepth is the
// per-frame channel count (3 for RGB) — RIFE concatenates the two input
// frames internally to feed a 6-channel tensor into the flow network.
// The previous example here showed inputDepth: 6 which recreated the
// exact slicing bug the constructor was patched to prevent.
var architecture = new NeuralNetworkArchitecture<double>(
    inputType: InputType.ThreeDimensional,
    taskType: NeuralNetworkTaskType.Regression,
    inputHeight: 256, inputWidth: 256, inputDepth: 3, outputSize: 3);
var model = new RIFE<double>(architecture, options: new RIFEOptions { NumFeatures = 64, NumFlowBlocks = 8 });

Constructors

Constructor Summary
RIFE() Initializes a new instance with default architecture settings.
RIFE(NeuralNetworkArchitecture<T> architecture, RIFEOptions options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer) Initializes a new instance of the RIFE class.

Properties

Property Summary
bool SupportsTraining { get; } Gets whether training is supported.

Methods

Method Summary
List<Tensor<T>> UpsampleFrameRate(List<Tensor<T>> frames, int factor) Interpolates multiple frames between input frames for frame rate upsampling.
ModelMetadata<T> GetModelMetadata()
Tensor<T> ForwardForTraining(Tensor<T> input)
Tensor<T> Interpolate(Tensor<T> frame1, Tensor<T> frame2, double timestep) Interpolates frames between two input frames.