M2MOptions

Options & Configuration · AiDotNet.Video.Options

Configuration options for M2M, many-to-many splatting for video frame interpolation (Hu, Niklaus, Sclaroff and Saenko, CVPR 2022).

Inheritance: ModelOptions → NeuralNetworkOptions → M2MOptions

For Beginners

M2M estimates rough motion once, then refines it into several motion guesses for every pixel. Each pixel is pushed forward along all of its guesses to the in-between time, and where several pixels land on the same spot they are blended by how reliable each one is. There is no heavy synthesis network, so it is fast.

How It Works

The defaults are the release’s M2M-PWC (feinanshan/M2M_VFI: Train/vfi/model/m2m.py, pwcnet.py, configs/vimeo90k.yml): PWC-Net flows at a quarter of the input size, a Motion Refinement Network of base width 16 that predicts four flows per pixel each way, and a rank-16 attention cube. Training follows the release: AdamW at 1e-4 with a weight decay of 1e-4, a 2,000-step linear warmup and cosine decay to zero over 900,000 steps, gradient norm clipped to 1, the flow estimator held at a learning rate of zero, on the Charbonnier plus ternary census loss.

Constructors

Constructor Summary
M2MOptions() Initializes a new instance with default values.
M2MOptions(M2MOptions other) Initializes a new instance by copying from another instance.

Properties

Property Summary
OnnxModelOptions OnnxOptions { get; set; } Gets or sets the ONNX runtime options.
double CharbonnierWeight { get; set; } Gets or sets the weight of the Charbonnier loss (1).
double FlowLearningRateMultiplier { get; set; } Gets or sets the flow estimator’s learning rate as a multiple of the network’s (0: the release loads a pretrained PWC-Net and holds it fixed).
double LearningRate { get; set; } Gets or sets AdamW’s peak learning rate (lr0, 1e-4).
double MaxGradientNorm { get; set; } Gets or sets the gradient norm clip (1).
double TernaryWeight { get; set; } Gets or sets the weight of the ternary census loss (1).
double WeightDecay { get; set; } Gets or sets AdamW’s decoupled weight decay (wd, 1e-4).
int BaseChannels { get; set; } Gets or sets the Motion Refinement Network’s base width c (16); its widths run from c to 16c.
int CubeRank { get; set; } Gets or sets the rank of the channel-height-width attention cube at 1/16 size (16).
int FlowChannels { get; set; } Gets or sets PWC-Net’s feature width at every pyramid level (32).
int NumBranches { get; set; } Gets or sets the flows predicted per pixel in each direction (branch, 4).
int Ratio { get; set; } Gets or sets the reduction, a power of two of at least 2, at which PWC-Net estimates the coarse flows (ratio, 4).
int TotalSteps { get; set; } Gets or sets the step at which the cosine decay reaches zero (max_iter, 900,000).
int WarmupSteps { get; set; } Gets or sets the steps over which the learning rate rises linearly from zero (warmup_steps, 2,000).
int[] FlowDecoderChannels { get; set; } Gets or sets the widths of each PWC-Net decoder’s five hidden convolutions (128, 128, 96, 64, 32).
string ModelPath { get; set; } Gets or sets the path to the ONNX model file.