SoftSplatOptions

Options & Configuration · AiDotNet.Video.Options

Configuration options for SoftSplat, softmax splatting for video frame interpolation (Niklaus and Liu, CVPR 2020).

Inheritance: ModelOptions → NeuralNetworkOptions → SoftSplatOptions

For Beginners

SoftSplat estimates motion both ways between the two frames, pushes each frame’s feature pyramid forward along its motion to the in-between time (blending pixels that land on the same spot by a learned importance), and then lets a grid of convolutions paint the in-between frame from the pushed features.

How It Works

The widths are the release’s (sniklaus/softmax-splatting, run.py): the PWC-Net flow estimator, the three-level importance-metric U-Net and the three-stream GridNet synthesis. Training follows the paper, which adopts the training of context-aware synthesis (Niklaus and Liu, CVPR 2018): AdaMax at 1e-3 on the five-level Laplacian loss.

Constructors

Constructor Summary
SoftSplatOptions() Initializes a new instance with default values.
SoftSplatOptions(SoftSplatOptions other) Initializes a new instance by copying from another instance.

Properties

Property Summary
OnnxModelOptions OnnxOptions { get; set; } Gets or sets the ONNX runtime options.
double Beta1 { get; set; } Gets or sets AdaMax’s first-moment decay (0.9).
double Beta2 { get; set; } Gets or sets AdaMax’s infinity-norm decay (0.999).
double LearningRate { get; set; } Gets or sets AdaMax’s learning rate (1e-3).
int LaplacianLevels { get; set; } Gets or sets the Laplacian loss’s pyramid levels, the last being the low-pass residual (5).
int[] FlowChannels { get; set; } Gets or sets the flow estimator’s six pyramid widths, 1/2 to 1/64 size (16, 32, 64, 96, 128, 192).
int[] FlowDecoderChannels { get; set; } Gets or sets the widths of each flow decoder’s five hidden convolutions (128, 128, 96, 64, 32).
int[] MetricChannels { get; set; } Gets or sets the importance-metric U-Net’s four row widths (16, 32, 64, 96); the first exceeds 4.
int[] SynthesisChannels { get; set; } Gets or sets the feature pyramid’s and the GridNet’s three stream widths (32, 64, 96).
string ModelPath { get; set; } Gets or sets the path to the ONNX model file.