MIAVSROptions
Options & Configuration · AiDotNet.Video.Options
Configuration options for MIA-VSR, the masked inter- and intra-frame attention video super-resolution transformer (Zhou et al., CVPR 2024).
Inheritance: ModelOptions → NeuralNetworkOptions → MIAVSROptions
For Beginners
MIA-VSR upscales a video one frame at a time while looking back at the frames it has already enhanced. Each attention block learns a mask that says which positions changed since the previous frame; positions that did not change reuse last frame’s result instead of being recomputed, which is where the method saves its compute.
How It Works
The defaults are the paper’s: 120 channels, 8x8 windows, 6 heads, an FFN ratio of 2, four propagation branches (backward, forward, backward, forward) of six inter-and-intra-frame attention blocks each, and a mask-sparsity weight of 5e-4 with a Gumbel-softmax temperature of 2/3.
Constructors
| Constructor | Summary |
|---|---|
MIAVSROptions() |
Initializes a new instance with default values. |
MIAVSROptions(MIAVSROptions other) |
Initializes a new instance by copying from another instance. |
Properties
| Property | Summary |
|---|---|
OnnxModelOptions OnnxOptions { get; set; } |
Gets or sets the ONNX runtime options. |
VideoModelVariant Variant { get; set; } |
Gets or sets the model variant. |
bool FineTuneFlowEstimator { get; set; } |
Gets or sets whether a supplied pretrained SPyNet is fine-tuned during training. |
double DropoutRate { get; set; } |
Gets or sets the dropout rate. |
double FlowLearningRateScale { get; set; } |
Gets or sets SPyNet’s learning rate as a fraction of the model’s while fine-tuning. |
double FlowLossWeight { get; set; } |
Gets or sets λ_flow, the weight of the photometric flow loss whenever SPyNet trains. |
double GumbelTemperature { get; set; } |
Gets or sets τ, the temperature of the training-time Gumbel-softmax mask. |
double InterMaskRatio { get; set; } |
Not used: MIA-VSR learns its masks; there is no fixed masking ratio. |
double IntraMaskRatio { get; set; } |
Not used: MIA-VSR learns its masks; there is no fixed masking ratio. |
double LearningRate { get; set; } |
Gets or sets the learning rate. |
double MaskLossWeight { get; set; } |
Gets or sets λ, the weight of the mask-sparsity loss added to the training objective. |
int BlocksPerBranch { get; set; } |
Gets or sets the number of inter-and-intra-frame attention blocks per branch. |
int FeedForwardRatio { get; set; } |
Gets or sets the FFN hidden width as a multiple of NumFeatures. |
int FlowFreezeSteps { get; set; } |
Gets or sets how many training steps a fine-tuned pretrained SPyNet stays frozen. |
int NumFeatures { get; set; } |
Gets or sets the feature width C of every attention block. |
int NumHeads { get; set; } |
Gets or sets the number of attention heads. |
int NumPropagationBranches { get; set; } |
Gets or sets the number of feature propagation modules (branches). |
int NumResBlocks { get; set; } |
Not used: MIA-VSR has no residual convolution blocks. |
int ReconstructionChannels { get; set; } |
Gets or sets the channel width of the pixel-shuffle reconstruction head. |
int ScaleFactor { get; set; } |
Gets or sets the spatial upscaling factor (a power of two). |
int WindowSize { get; set; } |
Gets or sets the side of the square attention window. |
string ModelPath { get; set; } |
Gets or sets the path to the ONNX model file. |