TLBVFI<T>

Models & Types · AiDotNet.Video.FrameInterpolation

TLBVFI: token-level bidirectional video frame interpolation.

Inheritance: NeuralNetworkBase<T> → VideoNeuralNetworkBase<T> → FrameInterpolationBase<T> → TLBVFI<T>

For Beginners

TLBVFI works with small patches (tokens) instead of individual pixels, like reading words instead of individual letters. It matches patches in both frames and uses them to build the intermediate frame efficiently.

Usage:

How It Works

TLBVFI (2024) operates at the token level for bidirectional frame interpolation:

  • Token-level processing: divides frames into non-overlapping tokens and performs all

operations at the token level for efficient transformer-based processing

  • Bidirectional token matching: finds corresponding tokens in both input frames using

learned cross-attention, handling forward and backward motion in a single pass

  • Token-level flow: estimates optical flow at the patch level for robustness and efficiency,

with sub-token refinement after initial matching

  • Adaptive token merging: dynamically merges tokens in low-motion regions while keeping

fine-grained tokens in high-motion areas

Reference: “TLBVFI: Token-Level Bidirectional Video Frame Interpolation” (2024)

Constructors

Constructor Summary
TLBVFI(NeuralNetworkArchitecture<T> architecture, TLBVFIOptions options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer) Creates a TLBVFI model in native training mode.
TLBVFI(NeuralNetworkArchitecture<T> architecture, string modelPath, TLBVFIOptions options) Creates a TLBVFI model in native training mode.

Methods

Method Summary
ModelMetadata<T> GetModelMetadata()
Tensor<T> Interpolate(Tensor<T> frame0, Tensor<T> frame1, double t)