GIMMVFI<T>

Models & Types · AiDotNet.Video.FrameInterpolation

GIMM-VFI: generalizable implicit motion modeling for video frame interpolation.

Inheritance: NeuralNetworkBase<T> → VideoNeuralNetworkBase<T> → FrameInterpolationBase<T> → GIMMVFI<T>

For Beginners

GIMM-VFI learns a smooth “motion field” that describes how everything in the scene moves over time. Once it processes two frames, it can generate a new frame at ANY point between them. This is great for variable slow-motion effects or non-uniform frame rate conversion where you need different time spacings between frames.

Usage:

How It Works

GIMM-VFI (NeurIPS 2024) uses implicit neural representations for continuous-time motion:

  • Implicit motion function: learns a continuous function M(x, y, t) that maps any spatial

position (x, y) and any timestep t in [0, 1] to a motion vector, enabling interpolation at arbitrary time intervals without retraining or additional forward passes

  • Motion encoding network: encodes two input frames into a shared motion latent space

using cross-correlation features, producing a motion representation that can be queried at any desired timestep

  • Generalizable across timesteps: a single forward pass through the motion encoder produces

a representation that the implicit function can query at any t, unlike methods that need separate inference per timestep

  • Adaptive sampling: the implicit function can be queried at higher density in regions with

complex motion and lower density in static regions for efficient computation

Reference: “GIMM-VFI: Generalizable Implicit Motion Modeling for Video Frame Interpolation” (NeurIPS 2024)

Constructors

Constructor Summary
GIMMVFI(NeuralNetworkArchitecture<T> architecture, GIMMVFIOptions options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer) Creates a GIMM-VFI model in native training mode.
GIMMVFI(NeuralNetworkArchitecture<T> architecture, string modelPath, GIMMVFIOptions options) Creates a GIMM-VFI model in native training mode.

Methods

Method Summary
ModelMetadata<T> GetModelMetadata()
Tensor<T> Interpolate(Tensor<T> frame0, Tensor<T> frame1, double t)