RealViformerOptions
Options & Configuration · AiDotNet.Video.Options
Configuration options for RealViformer, the channel-attention recurrent transformer for real-world video super-resolution.
Inheritance: ModelOptions → NeuralNetworkOptions → RealViformerOptions
For Beginners
RealViformer upscales real, messy video four times. These settings control how wide and deep the network is and how it is trained: first to match the clean frames, then to also look realistic.
How It Works
Defaults follow Zhang and Yao, ECCV 2024, Sec. 4.4, and the authors’ release (Yuehan717/RealViformer, inference_realviformer.py): 48 channels, [2, 3, 4] transformer blocks over three levels plus one refinement block, [1, 2, 4] heads, GDFN expansion 2.66, a two-head CAF, attention squeezed by 4 with the channel mask, 15-frame training clips, SPyNet frozen for 5,000 iterations, 300,000 iterations of Charbonnier + SSIM and then perceptual and GAN losses (weights 1, 0.001, 1, 0.005) with RealBasicVSR’s discriminator. The optimizer is not in the public paper, so the rates are RealBasicVSR’s: Adam (0.9, 0.99) at 1e-4, then 5e-5 for the generator and 1e-4 for the discriminator.
Constructors
| Constructor | Summary |
|---|---|
RealViformerOptions() |
Initializes a new instance with default values. |
RealViformerOptions(RealViformerOptions other) |
Initializes a new instance by copying from another instance. |
Properties
| Property | Summary |
|---|---|
OnnxModelOptions OnnxOptions { get; set; } |
Gets or sets the ONNX runtime options. |
bool ChannelMask { get; set; } |
Gets or sets whether the attention re-weights its channels with the learned mask (true). |
double AdversarialLearningRate { get; set; } |
Gets or sets the second stage’s generator Adam rate (5e-5). |
double CharbonnierEpsilon { get; set; } |
Gets or sets the Charbonnier ε (1e-6, i.e. |
double CharbonnierWeight { get; set; } |
Gets or sets the Charbonnier loss weight (1). |
double DiscriminatorLearningRate { get; set; } |
Gets or sets the discriminator’s Adam rate (1e-4). |
double FfnExpansionFactor { get; set; } |
Gets or sets the GDFN’s hidden width as a multiple of the channels (2.66). |
double GANWeight { get; set; } |
Gets or sets the GAN loss weight in the second stage (0.005). |
double LearningRate { get; set; } |
Gets or sets the first stage’s Adam rate (1e-4). |
double PerceptualWeight { get; set; } |
Gets or sets the perceptual loss weight in the second stage (1; 0 leaves the VGG19 out). |
double SsimWeight { get; set; } |
Gets or sets the SSIM loss weight (0.001). |
double? EmaMomentum { get; set; } |
Gets or sets the momentum of a moving average of the weights used for inference; null (the paper reports none) uses the trained weights. |
int DiscriminatorChannels { get; set; } |
Gets or sets the discriminator’s base width (64, RealBasicVSR’s U-Net discriminator). |
int FlowFreezeIterations { get; set; } |
Gets or sets the iterations SPyNet stays fixed at the start of training (5,000). |
int MergeHeads { get; set; } |
Gets or sets the heads of the CAF that merges the hidden state (2). |
int NumFeatures { get; set; } |
Gets or sets the first level’s channels (48; the levels below have 96 and 192). |
int NumFrames { get; set; } |
Gets or sets the frames of a training clip (15). |
int ReconstructionIterations { get; set; } |
Gets or sets the iterations of the Charbonnier + SSIM stage before perceptual and GAN losses join (300,000). |
int SqueezeFactor { get; set; } |
Gets or sets the factor the attention squeezes channels by (4; 1 disables the squeeze). |
int[] Heads { get; set; } |
Gets or sets the attention heads of levels 1-3 ([1, 2, 4]). |
int[] NumBlocks { get; set; } |
Gets or sets the transformer blocks of levels 1-3 and the refinement ([2, 3, 4, 1]). |
string ModelPath { get; set; } |
Gets or sets the path to the ONNX model file. |