MoMo<T>
Models & Types · AiDotNet.Video.FrameInterpolation
MoMo: momentum diffusion model for bi-directional optical flow in frame interpolation.
Inheritance: NeuralNetworkBase<T> → VideoNeuralNetworkBase<T> → FrameInterpolationBase<T> → MoMo<T>
For Beginners
Most frame interpolation methods estimate motion (optical flow) using direct prediction, which can be blurry at object edges. MoMo instead uses a generative AI model to create sharper, more accurate motion fields, then uses those flows to produce the intermediate frame. Think of it as using AI to “draw” better motion maps.
Usage:
How It Works
MoMo (2024) is the first diffusion model for bi-directional optical flow in VFI:
- Flow diffusion: instead of directly regressing optical flow from a CNN (which produces
over-smoothed flow at boundaries), MoMo uses a denoising diffusion model to generate bi-directional flow fields, capturing sharper motion boundaries and multi-modal flow
- Momentum-based flow modeling: incorporates a momentum prior that biases flow generation
toward physically plausible motions, reducing artifacts from unrealistic flow predictions
- Joint bi-directional generation: generates forward (t0 to t) and backward (t1 to t)
flows simultaneously in a single diffusion process, ensuring temporal consistency between the two flow fields
- Flow-to-frame synthesis: the generated flows are used for backward warping with learned
occlusion masks and residual refinement to produce the final interpolated frame
Reference: “MoMo: Momentum Diffusion Model for Bi-Directional Flow” (2024)
Constructors
| Constructor | Summary |
|---|---|
MoMo(NeuralNetworkArchitecture<T> architecture, MoMoOptions options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer) |
Creates a MoMo model in native training mode. |
MoMo(NeuralNetworkArchitecture<T> architecture, string modelPath, MoMoOptions options) |
Creates a MoMo model in native training mode. |
Methods
| Method | Summary |
|---|---|
ModelMetadata<T> GetModelMetadata() |
|
Tensor<T> Interpolate(Tensor<T> frame0, Tensor<T> frame1, double t) |