RVRTOptions

Options & Configuration · AiDotNet.Video.Options

Configuration options for RVRT, the recurrent video restoration transformer with guided deformable attention.

Inheritance: ModelOptions → NeuralNetworkOptions → RVRTOptions

For Beginners

RVRT splits a video into pairs of frames and walks through them backward and forward in time, carrying what it learned from earlier pairs. To line up a pair with the previous one it follows the estimated motion and then looks around each moved point for the best-matching detail.

How It Works

Defaults follow Liang et al., NeurIPS 2022, and the authors’ release (JingyunLiang/RVRT network_rvrt.py and KAIR options/rvrt/001_train_rvrt_videosr_bi_reds_30frames.json): 144-channel embeddings, 6 attention heads, 2x8x8 windows, RSTB groups of depth 2 ([1, 2, 1] blocks for extraction, each propagation pass and reconstruction), guided deformable attention with 12 groups and 12 heads over 3x3 windows and offset residues up to 10 pixels, two-frame clips, 30-frame training sequences, Charbonnier loss (eps 1e-9 under the square root), and Adam (0.9, 0.99) at 4e-4 with SPyNet at 0.25 times the rate and frozen for the first 30,000 iterations, cosine-annealed to 1e-7 over 600,000 iterations.

Constructors

Constructor Summary
RVRTOptions() Initializes a new instance with default values.
RVRTOptions(RVRTOptions other) Initializes a new instance by copying from another instance.

Properties

Property Summary
OnnxModelOptions OnnxOptions { get; set; } Gets or sets the ONNX runtime options.
double CharbonnierEpsilon { get; set; } Gets or sets the Charbonnier ε; KAIR’s loss is sqrt(d² + 1e-9), so ε = sqrt(1e-9).
double LearningRate { get; set; } Gets or sets Adam’s starting learning rate (4e-4).
double MaxResidueMagnitude { get; set; } Gets or sets the largest offset residue on top of the flow, in pixels (10).
double MinLearningRate { get; set; } Gets or sets the floor of the cosine schedule (1e-7).
double MlpRatio { get; set; } Gets or sets the MLP width as a multiple of the embedding (2).
double SpyNetLearningRateMultiplier { get; set; } Gets or sets the multiplier of SPyNet’s learning rate (0.25, i.e.
int AttentionHeads { get; set; } Gets or sets the heads of the guided deformable attention (12).
int CosineIterations { get; set; } Gets or sets the iterations of the cosine schedule (600,000).
int DeformableGroups { get; set; } Gets or sets the deformable offset groups of the guided deformable attention (12).
int Depth { get; set; } Gets or sets the Swin layers per residual block (2).
int EmbedDim { get; set; } Gets or sets the embedding width of every stage (144).
int ExtractionBlocks { get; set; } Gets or sets the residual Swin blocks of the feature extractor (1).
int FlowFreezeIterations { get; set; } Gets or sets the iterations SPyNet stays frozen at the start of training (30,000).
int NumFrames { get; set; } Gets or sets the frames of a training sequence (30, the REDS recipe; a multiple of the clip size 2).
int NumHeads { get; set; } Gets or sets the window-attention heads (6).
int PropagationBlocks { get; set; } Gets or sets the residual Swin blocks of each propagation pass (2).
int ReconstructionBlocks { get; set; } Gets or sets the residual Swin blocks of the reconstruction (1).
int WindowSize { get; set; } Gets or sets the spatial window size (8; windows are 2x8x8 in propagation, 1x8x8 elsewhere).
string ModelPath { get; set; } Gets or sets the path to the ONNX model file.