SwinVFI<T>

Models & Types · AiDotNet.Video.FrameInterpolation

SwinVFI: Swin Transformer-based video frame interpolation.

Inheritance: NeuralNetworkBase<T> → VideoNeuralNetworkBase<T> → FrameInterpolationBase<T> → SwinVFI<T>

For Beginners

SwinVFI uses the Swin Transformer to look at both input frames simultaneously and figure out what goes between them, without needing to estimate motion. The “shifted window” approach makes it efficient for full-resolution video frames.

Usage:

How It Works

SwinVFI (2022) applies Swin Transformer architecture to frame interpolation:

  • Swin Transformer encoder: uses shifted-window self-attention with linear complexity for

encoding input frame pairs at high resolution

  • Cross-frame window attention: extends shifted-window attention to cross-attend between

features from both input frames, capturing inter-frame correspondences

  • Hierarchical feature pyramid: multi-scale feature extraction with Swin blocks at each level
  • Flow-free synthesis: directly synthesizes the intermediate frame from cross-attended

features without explicit optical flow estimation

Reference: “SwinVFI: Swin Transformer-based Video Frame Interpolation” (2022)

Constructors

Constructor Summary
SwinVFI(NeuralNetworkArchitecture<T> architecture, SwinVFIOptions options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer) Creates a SwinVFI model in native training mode.
SwinVFI(NeuralNetworkArchitecture<T> architecture, string modelPath, SwinVFIOptions options) Creates a SwinVFI model in native training mode.

Methods

Method Summary
ModelMetadata<T> GetModelMetadata()
Tensor<T> Interpolate(Tensor<T> frame0, Tensor<T> frame1, double t)