FILM<T>

Models & Types · AiDotNet.Video.FrameInterpolation

FILM (Frame Interpolation for Large Motion) model for high-quality frame interpolation.

Inheritance: NeuralNetworkBase<T> → VideoNeuralNetworkBase<T> → FrameInterpolationBase<T> → FILM<T>

For Beginners

FILM generates smooth intermediate frames between two input frames, even when there’s significant motion between them. It’s particularly good at:

  • Large motion scenes (fast camera movements, rapid object motion)
  • Creating slow-motion effects from regular video
  • Increasing video frame rate (24fps to 60fps)
  • Smooth transitions between keyframes

Unlike older methods that struggle with large motions, FILM uses a multi-scale feature extraction approach that handles both small and large movements gracefully.

How It Works

Technical Details:

  • Multi-scale feature pyramid for handling large motions
  • Bi-directional flow estimation with occlusion handling
  • Feature-based frame synthesis (not just flow warping)
  • Scale-agnostic architecture for arbitrary resolution

Reference: Reda et al., “FILM: Frame Interpolation for Large Motion” ECCV 2022.

Example

// Create a FILM model for frame interpolation with large motion handling
var film = new FILM<double>();
            
// Or configure with custom multi-scale feature pyramid settings
var architecture = new NeuralNetworkArchitecture<double>(
    inputType: InputType.ThreeDimensional,
    taskType: NeuralNetworkTaskType.Regression,
    inputHeight: 256, inputWidth: 256, inputDepth: 6, outputSize: 3);
var model = new FILM<double>(architecture, options: new FILMOptions { NumScales = 7, NumFeatures = 64 });

Constructors

Constructor Summary
FILM() Initializes a new instance of the FILM class.
FILM(NeuralNetworkArchitecture<T> architecture, FILMOptions options)

Properties

Property Summary
bool SupportsTraining { get; } Gets whether training is supported.

Methods

Method Summary
List<Tensor<T>> CreateSlowMotion(List<Tensor<T>> frames, int slowdownFactor) Creates slow-motion effect from video frames.
List<Tensor<T>> IncreaseFrameRate(List<Tensor<T>> frames, int factor) Increases video frame rate by a given factor.
List<Tensor<T>> InterpolateMultiple(Tensor<T> frame1, Tensor<T> frame2, int numIntermediateFrames) Generates multiple intermediate frames between two input frames.
ModelMetadata<T> GetModelMetadata()
Tensor<T> ForwardForTraining(Tensor<T> input)
Tensor<T> Interpolate(Tensor<T> frame1, Tensor<T> frame2, double timestep) Interpolates a frame between two input frames.