VFITOptions
Options & Configuration · AiDotNet.Video.Options
Configuration options for VFIT, the video frame interpolation
transformer (Shi et al., CVPR 2022).
Inheritance: ModelOptions → NeuralNetworkOptions → VFITOptions
For Beginners
VFIT looks at two frames before and two after the missing one. A transformer compares small windows within each frame and the same pixel across the four frames, and the result decides, for every output pixel, which nearby input pixels to blend and by how much.
How It Works
The defaults are the release’s VFIT-B (zhshi0816/Video-Frame-Interpolation-Transformer: model/VFIT_B.py,
Sep_STS_Encoder.py, Sep_STS_Layer.py, main.py, config.py): four input frames, stage widths 64, 128, 256 and 512 with
2, 2, 6 and 2 Sep-STS blocks and 2, 4, 8 and 16 heads, 8x8 spatial windows, an MLP ratio of 4, and 5x5 AdaCoF kernels
from 64-channel features. VFIT-S is the same with the widths 32, 64, 128 and 192
(Widths = { 192, 128, 64, 32 }). Training follows the release: Adamax (betas 0.9 and 0.999) on the L1
loss, its rate stepped through 2e-4, 1e-4, 5e-5, 2.5e-5, 5e-6 and 1e-6 at epochs 40, 60, 75, 85 and 95 of 100.
Constructors
| Constructor | Summary |
|---|---|
VFITOptions() |
Initializes a new instance with default values. |
VFITOptions(VFITOptions other) |
Initializes a new instance by copying from another instance. |
Properties
| Property | Summary |
|---|---|
OnnxModelOptions OnnxOptions { get; set; } |
Gets or sets the ONNX runtime options. |
double Beta1 { get; set; } |
Gets or sets Adamax’s first-moment decay (0.9). |
double Beta2 { get; set; } |
Gets or sets Adamax’s infinity-norm decay (0.999). |
double LearningRate { get; set; } |
Gets or sets Adamax’s initial learning rate (2e-4). |
double MlpRatio { get; set; } |
Gets or sets the MLP’s hidden width as a multiple of the stage width (4). |
double[] LearningRateSteps { get; set; } |
Gets or sets the rates after each milestone (1e-4, 5e-5, 2.5e-5, 5e-6, 1e-6). |
int Dilation { get; set; } |
Gets or sets the AdaCoF kernel dilation (1). |
int KernelSize { get; set; } |
Gets or sets the AdaCoF kernel side (ks, 5). |
int NumInputFrames { get; set; } |
Gets or sets the input frames, two before and two after the target (4). |
int SynthesisChannels { get; set; } |
Gets or sets the synthesis features’ width (nf_out, 64). |
int WindowSize { get; set; } |
Gets or sets the spatial attention window side (8); every second block shifts it by half. |
int[] LearningRateMilestones { get; set; } |
Gets or sets the batches at which the rate steps to the next of LearningRateSteps: epochs 40, 60, 75, 85 and 95 of the 16,153 batches of 4 in Vimeo-90K’s 64,612 training septuplets (646,120, 969,180, 1,211,475, 1,373,005, 1,534,535). |
int[] StageDepths { get; set; } |
Gets or sets the Sep-STS blocks of the four stages, shallowest first (2, 2, 6, 2). |
int[] StageHeads { get; set; } |
Gets or sets the attention heads of the four stages, shallowest first (2, 4, 8, 16). |
int[] Widths { get; set; } |
Gets or sets the stage widths, deepest first (nf: 512, 256, 128, 64 for VFIT-B; 192, 128, 64, 32 for VFIT-S); the stem is half the last. |
string ModelPath { get; set; } |
Gets or sets the path to the ONNX model file. |