AMTOptions

Options & Configuration · AiDotNet.Video.Options

Configuration options for AMT, all-pairs multi-field transforms for frame interpolation (Li et al., CVPR 2023).

Inheritance: ModelOptions → NeuralNetworkOptions → AMTOptions

For Beginners

AMT compares every pixel of one frame with every pixel of the other to find where things moved, predicts several candidate motions per pixel, and blends the frames moved along each into the in-between frame.

How It Works

The defaults are AMT-S, the release’s default configuration (MCG-NKU/AMT, networks/AMT-S.py, cfgs/AMT-S.yaml); AmtL returns AMT-L’s. Training follows trainers/base_trainer.py.

Constructors

Constructor Summary
AMTOptions() Initializes a new instance with default values.
AMTOptions(AMTOptions other) Initializes a new instance by copying from another instance.

Properties

Property Summary
OnnxModelOptions OnnxOptions { get; set; } Gets or sets the ONNX runtime options.
bool LargeEncoders { get; set; } Gets or sets whether the encoders are AMT-L’s: residual (not bottleneck) feature-encoder blocks and a 7x7 first context convolution (false).
double LearningRate { get; set; } Gets or sets AdamW’s initial learning rate (2e-4).
double MinLearningRate { get; set; } Gets or sets the rate the cosine schedule ends at (2e-5).
double TernaryWeight { get; set; } Gets or sets the ternary (census) loss’s weight against the Charbonnier term (1).
double WeightDecay { get; set; } Gets or sets AdamW’s weight decay (0).
int CombineKernelSize { get; set; } Gets or sets the kernel of the flow-combination convolutions (3).
int CorrelationRadius { get; set; } Gets or sets the correlation lookup radius (3).
int FeatureDim { get; set; } Gets or sets the correlation features’ width (84).
int NumCorrelationLevels { get; set; } Gets or sets the correlation pyramid’s levels (4).
int NumFlowFields { get; set; } Gets or sets the number of flow fields per pixel at full size (3).
int SkipChannels { get; set; } Gets or sets the ResBlocks’ side channels (20).
int TotalIterations { get; set; } Gets or sets the cosine schedule’s length: 300 epochs of Vimeo90K’s 51,312 triplets at batch 24.
int UpdateCorrelationDim { get; set; } Gets or sets the update blocks’ correlation-feature width (64).
int UpdateFlowDim { get; set; } Gets or sets the update blocks’ flow-feature width (20).
int UpdateFusionDim { get; set; } Gets or sets the update blocks’ fused motion-feature width (68).
int UpdateHiddenDim { get; set; } Gets or sets the update blocks’ hidden width (76).
int UpdateSecondCorrelationDim { get; set; } Gets or sets the width of the second correlation convolution, 0 for none (AMT-S’s SmallUpdateBlock).
int[] Channels { get; set; } Gets or sets the context encoder’s four widths, 1/2 to 1/16 size (20, 32, 44, 56).
int[] FeatureEncoderWidths { get; set; } Gets or sets the correlation feature encoder’s three widths (32, 64, 96).
string ModelPath { get; set; } Gets or sets the path to the ONNX model file.

Methods

Method Summary
AMTOptions AmtL() AMT-L (networks/AMT-L.py, cfgs/AMT-L.yaml): wider, five flow fields, residual encoders.