MIAVSROptions

Options & Configuration · AiDotNet.Video.Options

Configuration options for MIA-VSR, the masked inter- and intra-frame attention video super-resolution transformer (Zhou et al., CVPR 2024).

Inheritance: ModelOptions → NeuralNetworkOptions → MIAVSROptions

For Beginners

MIA-VSR upscales a video one frame at a time while looking back at the frames it has already enhanced. Each attention block learns a mask that says which positions changed since the previous frame; positions that did not change reuse last frame’s result instead of being recomputed, which is where the method saves its compute.

How It Works

The defaults are the paper’s: 120 channels, 8x8 windows, 6 heads, an FFN ratio of 2, four propagation branches (backward, forward, backward, forward) of six inter-and-intra-frame attention blocks each, and a mask-sparsity weight of 5e-4 with a Gumbel-softmax temperature of 2/3.

Constructors

Constructor Summary
MIAVSROptions() Initializes a new instance with default values.
MIAVSROptions(MIAVSROptions other) Initializes a new instance by copying from another instance.

Properties

Property Summary
OnnxModelOptions OnnxOptions { get; set; } Gets or sets the ONNX runtime options.
VideoModelVariant Variant { get; set; } Gets or sets the model variant.
bool FineTuneFlowEstimator { get; set; } Gets or sets whether a supplied pretrained SPyNet is fine-tuned during training.
double DropoutRate { get; set; } Gets or sets the dropout rate.
double FlowLearningRateScale { get; set; } Gets or sets SPyNet’s learning rate as a fraction of the model’s while fine-tuning.
double FlowLossWeight { get; set; } Gets or sets λ_flow, the weight of the photometric flow loss whenever SPyNet trains.
double GumbelTemperature { get; set; } Gets or sets τ, the temperature of the training-time Gumbel-softmax mask.
double InterMaskRatio { get; set; } Not used: MIA-VSR learns its masks; there is no fixed masking ratio.
double IntraMaskRatio { get; set; } Not used: MIA-VSR learns its masks; there is no fixed masking ratio.
double LearningRate { get; set; } Gets or sets the learning rate.
double MaskLossWeight { get; set; } Gets or sets λ, the weight of the mask-sparsity loss added to the training objective.
int BlocksPerBranch { get; set; } Gets or sets the number of inter-and-intra-frame attention blocks per branch.
int FeedForwardRatio { get; set; } Gets or sets the FFN hidden width as a multiple of NumFeatures.
int FlowFreezeSteps { get; set; } Gets or sets how many training steps a fine-tuned pretrained SPyNet stays frozen.
int NumFeatures { get; set; } Gets or sets the feature width C of every attention block.
int NumHeads { get; set; } Gets or sets the number of attention heads.
int NumPropagationBranches { get; set; } Gets or sets the number of feature propagation modules (branches).
int NumResBlocks { get; set; } Not used: MIA-VSR has no residual convolution blocks.
int ReconstructionChannels { get; set; } Gets or sets the channel width of the pixel-shuffle reconstruction head.
int ScaleFactor { get; set; } Gets or sets the spatial upscaling factor (a power of two).
int WindowSize { get; set; } Gets or sets the side of the square attention window.
string ModelPath { get; set; } Gets or sets the path to the ONNX model file.