BasicVSRPlusPlus<T>
Models & Types · AiDotNet.Video.Enhancement
BasicVSR++ (Basic Video Super-Resolution++) for temporal video super-resolution.
Inheritance: NeuralNetworkBase<T> → VideoNeuralNetworkBase<T> → VideoSuperResolutionBase<T> → BasicVSRPlusPlus<T>
For Beginners
BasicVSR++ is a video super-resolution model that upscales low-resolution videos to higher resolution while maintaining temporal consistency. Unlike single-image methods (like RealESRGAN), it uses information from multiple frames to produce sharper and more consistent results.
Key concepts:
- Bidirectional Propagation: Uses both past and future frames to enhance
the current frame, ensuring temporal coherence.
- Optical Flow: Estimates how pixels move between frames to align features.
- Deformable Alignment: Uses learned offsets to precisely align features
even with complex motions.
Example usage:
How It Works
BasicVSR++ (Chan et al. 2022) improves upon BasicVSR with:
- Second-order grid propagation for better temporal modeling
- Flow-guided deformable alignment for accurate feature alignment
- Bidirectional propagation for utilizing both past and future frames
Implementation fidelity note: This native implementation realizes flow-guided
deformable alignment and SECOND-ORDER bidirectional grid propagation faithfully and
trains them end-to-end through the autodiff tape (see Train). Each
propagation step aggregates BOTH the immediately adjacent frame (i±1) and the second
neighbour (i±2), with the i→i±2 flow formed by COMPOSING the two adjacent flows
(Chan et al. 2022, Sec. 3.1); the deformable-alignment layer resolves its widened
(current + i±1 + i±2) input-channel count lazily on the first forward. One
simplification remains: the SPyNet flow estimator acts as a fixed sampling guide — its
warp keeps the sampled features on the tape (gradients reach the reconstruction
network) but SPyNet’s own weights are not fine-tuned here, matching the common
“pre-trained flow” setup rather than the paper’s fully joint training.
Reference: Chan et al., “BasicVSR++: Improving Video Super-Resolution with
Enhanced Propagation and Alignment”, CVPR 2022. https://arxiv.org/abs/2104.13371
Constructors
| Constructor | Summary |
|---|---|
BasicVSRPlusPlus() |
Initializes a new instance with default architecture settings. |
BasicVSRPlusPlus(NeuralNetworkArchitecture<T> architecture, BasicVSRPlusPlusOptions options) |
Creates a BasicVSR++ model for native training and inference. |
BasicVSRPlusPlus(NeuralNetworkArchitecture<T> architecture, string onnxModelPath, BasicVSRPlusPlusOptions options) |
Creates a BasicVSR++ model using a pretrained ONNX model for inference. |
Properties
| Property | Summary |
|---|---|
bool SupportsTraining { get; } |
Gets whether training is supported (only in native mode). |
Methods
| Method | Summary |
|---|---|
Dictionary<string, Tensor<T>> GetNamedLayerActivations(Tensor<T> input) |
Captures intermediate activations from BasicVSR++’s real recurrent forward pass for inspection/debugging. |
ModelMetadata<T> GetModelMetadata() |
|
Tensor<T> EnhanceVideo(Tensor<T> frames) |
Enhances a sequence of video frames using temporal super-resolution. |
Tensor<T> ForwardForTraining(Tensor<T> input) |
|
Tensor<T> Upscale(Tensor<T> lowResFrames) |