BasicVSRPlusPlus<T>

Models & Types · AiDotNet.Video.Enhancement

BasicVSR++ (Basic Video Super-Resolution++) for temporal video super-resolution.

Inheritance: NeuralNetworkBase<T> → VideoNeuralNetworkBase<T> → VideoSuperResolutionBase<T> → BasicVSRPlusPlus<T>

For Beginners

BasicVSR++ is a video super-resolution model that upscales low-resolution videos to higher resolution while maintaining temporal consistency. Unlike single-image methods (like RealESRGAN), it uses information from multiple frames to produce sharper and more consistent results.

Key concepts:

  1. Bidirectional Propagation: Uses both past and future frames to enhance

the current frame, ensuring temporal coherence.

  1. Optical Flow: Estimates how pixels move between frames to align features.
  2. Deformable Alignment: Uses learned offsets to precisely align features

even with complex motions.

Example usage:

How It Works

BasicVSR++ (Chan et al. 2022) improves upon BasicVSR with:

  • Second-order grid propagation for better temporal modeling
  • Flow-guided deformable alignment for accurate feature alignment
  • Bidirectional propagation for utilizing both past and future frames

Implementation fidelity note: This native implementation realizes flow-guided deformable alignment and SECOND-ORDER bidirectional grid propagation faithfully and trains them end-to-end through the autodiff tape (see Train). Each propagation step aggregates BOTH the immediately adjacent frame (i±1) and the second neighbour (i±2), with the i→i±2 flow formed by COMPOSING the two adjacent flows (Chan et al. 2022, Sec. 3.1); the deformable-alignment layer resolves its widened (current + i±1 + i±2) input-channel count lazily on the first forward. One simplification remains: the SPyNet flow estimator acts as a fixed sampling guide — its warp keeps the sampled features on the tape (gradients reach the reconstruction network) but SPyNet’s own weights are not fine-tuned here, matching the common “pre-trained flow” setup rather than the paper’s fully joint training.

Reference: Chan et al., “BasicVSR++: Improving Video Super-Resolution with Enhanced Propagation and Alignment”, CVPR 2022. https://arxiv.org/abs/2104.13371

Constructors

Constructor Summary
BasicVSRPlusPlus() Initializes a new instance with default architecture settings.
BasicVSRPlusPlus(NeuralNetworkArchitecture<T> architecture, BasicVSRPlusPlusOptions options) Creates a BasicVSR++ model for native training and inference.
BasicVSRPlusPlus(NeuralNetworkArchitecture<T> architecture, string onnxModelPath, BasicVSRPlusPlusOptions options) Creates a BasicVSR++ model using a pretrained ONNX model for inference.

Properties

Property Summary
bool SupportsTraining { get; } Gets whether training is supported (only in native mode).

Methods

Method Summary
Dictionary<string, Tensor<T>> GetNamedLayerActivations(Tensor<T> input) Captures intermediate activations from BasicVSR++’s real recurrent forward pass for inspection/debugging.
ModelMetadata<T> GetModelMetadata()
Tensor<T> EnhanceVideo(Tensor<T> frames) Enhances a sequence of video frames using temporal super-resolution.
Tensor<T> ForwardForTraining(Tensor<T> input)
Tensor<T> Upscale(Tensor<T> lowResFrames)