MGLDVSROptions
Options & Configuration · AiDotNet.Video.Options
Configuration options for MGLD-VSR, motion-guided latent diffusion for real-world video super-resolution.
Inheritance: ModelOptions → NeuralNetworkOptions → MGLDVSROptions
For Beginners
MGLD-VSR sharpens video with an image diffusion model. While it removes noise it checks how objects move between frames (optical flow) and nudges the frames to agree, so the result does not flicker.
How It Works
Defaults follow Yang et al. 2024 (ECCV, arXiv 2312.00853) and the release (IanYeung/MGLD-VSR,
mgldvsr_512_realbasicvsr_deg.yaml and video_autoencoder_kl_64x64x4_resi.yaml): Stable Diffusion 2.1-512
as the denoiser (320 wide, multipliers 1, 2, 4, 4, two residual blocks a level, 64-wide heads, a 1,024-wide OpenCLIP
context of 77 tokens) with SPADE on a 256-channel time-aware structure condition (256 wide, multipliers 1, 1, 2, 2,
four heads) and temporal modules in the middle block over five-frame clips; the linear DDPM schedule (0.00085 to
0.012 over 1,000 steps); SD’s KL autoencoder (128 wide, multipliers 1, 2, 4, 4) whose decoder gains CFW modules and
temporal convolutions; RAFT for the motion guidance at scale −10; and the decoder’s losses (L1, LPIPS 0.5, the frame
difference 0.5, the edge-weighted warping loss 0.5, and a PatchGAN weighted 0.025 adaptively from step 501).
Constructors
| Constructor | Summary |
|---|---|
MGLDVSROptions() |
Initializes a new instance with default values. |
MGLDVSROptions(MGLDVSROptions other) |
Initializes a new instance by copying from another instance. |
Properties
| Property | Summary |
|---|---|
OnnxModelOptions OnnxOptions { get; set; } |
Gets or sets the ONNX runtime options. |
bool ColorFix { get; set; } |
Gets or sets whether each output frame’s color statistics are matched to the LQ frame (AdaIN, the paper’s). |
double BetaEnd { get; set; } |
Gets or sets the last β (0.012). |
double BetaStart { get; set; } |
Gets or sets the first β of the linear (square-root-spaced) schedule (0.00085). |
double DecoderLearningRate { get; set; } |
Gets or sets the decoder stage’s Adam learning rate for the decoder and the discriminator (5e-5, β 0.5 and 0.9). |
double DifferenceWeight { get; set; } |
Gets or sets the decoder’s frame-difference loss weight (0.5). |
double DiscriminatorWeight { get; set; } |
Gets or sets the adversarial weight on the adaptive GAN weight (0.025). |
double EdgeWeight { get; set; } |
Gets or sets the warping loss’s edge weight w in 1 + w·Sobel(ground truth) (3). |
double FusionWeight { get; set; } |
Gets or sets the CFW fusion weight at inference (the release’s dec_w, 0.5). |
double GuidanceScale { get; set; } |
Gets or sets the motion guidance scale g (−10): after each step the latents move by −g·log σ²_t·∇L, with L the occlusion-masked warping error between neighbouring latents. |
double LatentScale { get; set; } |
Gets or sets the latent scale factor (SD’s 0.18215). |
double LearningRate { get; set; } |
Gets or sets the diffusion stage’s AdamW learning rate (the config’s base rate, 5e-5). |
double PerceptualWeight { get; set; } |
Gets or sets the decoder’s LPIPS weight (0.5). |
double TemporalWeight { get; set; } |
Gets or sets the decoder’s warping loss weight (0.5). |
double TrainingFusionWeight { get; set; } |
Gets or sets the CFW fusion weight while the decoder trains (1.0). |
int AttentionLevels { get; set; } |
Gets or sets how many leading levels carry spatial transformers (attention resolutions 4, 2, 1: three). |
int ContextDim { get; set; } |
Gets or sets the text context’s width (OpenCLIP ViT-H penultimate layer: 1,024). |
int ContextLength { get; set; } |
Gets or sets the text context’s token count (77). |
int DiffusionIterations { get; set; } |
Gets or sets the diffusion stage’s iterations before the decoder stage. |
int DiscriminatorStart { get; set; } |
Gets or sets the decoder step the adversarial loss starts at (501). |
int FlowIterations { get; set; } |
Gets or sets RAFT’s refinement iterations (RAFT_SR’s 10). |
int FuseBlocks { get; set; } |
Gets or sets the residual dense blocks in each CFW module (2). |
int HeadChannels { get; set; } |
Gets or sets the attention head width (64). |
int MinimumSide { get; set; } |
Gets or sets the shortest side the frames are upsampled to before diffusion (512, SD 2.1’s resolution); the result is resized back to the scale factor. |
int NumFrames { get; set; } |
Gets or sets the frames a clip the temporal modules span (5). |
int NumTrainTimesteps { get; set; } |
Gets or sets the DDPM training timesteps (1,000). |
int SamplingSteps { get; set; } |
Gets or sets the DDPM sampling steps, evenly respaced (the release’s default: 1,000). |
int ScaleFactor { get; set; } |
Gets or sets the upscaling factor (4). |
int StructureChannels { get; set; } |
Gets or sets the structure encoder’s width and its condition’s channels (256). |
int StructureHeads { get; set; } |
Gets or sets the structure encoder’s attention heads (4). |
int StructureResBlocks { get; set; } |
Gets or sets the structure encoder’s residual blocks a level (2). |
int UNetChannels { get; set; } |
Gets or sets the UNet’s base width (SD 2.1: 320). |
int UNetResBlocks { get; set; } |
Gets or sets the residual blocks a UNet level (2). |
int VaeChannels { get; set; } |
Gets or sets the KL autoencoder’s base width (128). |
int VaeResBlocks { get; set; } |
Gets or sets the encoder’s residual blocks a level (2; the decoder has one more). |
int[] StructureMultipliers { get; set; } |
Gets or sets the structure encoder’s width multipliers (1, 1, 2, 2). |
int[] UNetMultipliers { get; set; } |
Gets or sets the UNet’s width multipliers (1, 2, 4, 4). |
int[] VaeMultipliers { get; set; } |
Gets or sets the autoencoder’s width multipliers (1, 2, 4, 4; x8 compression). |
string ModelPath { get; set; } |
Gets or sets the ONNX model path. |