STMFNet<T>

Models & Types · AiDotNet.Video.FrameInterpolation

STMFNet: spatio-temporal multi-flow network for video frame interpolation.

Inheritance: NeuralNetworkBase<T> → VideoNeuralNetworkBase<T> → FrameInterpolationBase<T> → STMFNet<T>

For Beginners

STMFNet makes multiple motion guesses (flows) and picks the best one for each part of the image. This handles tricky areas like object boundaries much better than methods that make only one motion guess.

Usage:

How It Works

STMFNet (2022) uses multiple optical flows in spatio-temporal space:

  • Multi-flow estimation: estimates multiple (typically 4) optical flow fields, each capturing

different motion hypotheses for ambiguous regions

  • Spatio-temporal feature volume: constructs a 4D feature volume from input frames and all

estimated flow fields, capturing the full motion context

  • Flow selection network: selects the best flow hypothesis for each pixel by comparing

warped features from each flow field

  • Residual refinement: after flow-based warping, corrects remaining artifacts using the

multi-flow feature volume as context

Reference: “STMFNet: Spatio-Temporal Multi-Flow Network for Video Frame Interpolation” (2022)

Constructors

Constructor Summary
STMFNet(NeuralNetworkArchitecture<T> architecture, STMFNetOptions options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer) Creates an STMFNet model in native training mode.
STMFNet(NeuralNetworkArchitecture<T> architecture, string modelPath, STMFNetOptions options) Creates an STMFNet model in native training mode.

Methods

Method Summary
ModelMetadata<T> GetModelMetadata()
Tensor<T> Interpolate(Tensor<T> frame0, Tensor<T> frame1, double t)