StableVideoSR<T>

Models & Types · AiDotNet.Video.Enhancement

StableVideoSR: stable diffusion with temporal conditioning for video super-resolution.

Inheritance: NeuralNetworkBase<T> → VideoNeuralNetworkBase<T> → VideoSuperResolutionBase<T> → StableVideoSR<T>

For Beginners

StableVideoSR extends the popular Stable Diffusion image AI to handle video. It adds “temporal awareness” so the model considers what happened in previous and next frames when upscaling each frame, preventing the flickering that occurs when frames are processed independently.

Usage:

How It Works

StableVideoSR (2024) adapts the Stable Diffusion architecture for video SR:

  • Temporal conditioning modules: cross-attention layers inserted between spatial attention

in the U-Net attend to features from adjacent frames, maintaining temporal coherence

  • Direct low-resolution conditioning: noised RGB frames are concatenated with the latent

input, matching the released seven-channel U-Net contract

  • Classifier-free guidance: balances between faithful reconstruction and generative

enhancement during inference

  • Noise schedule: adapted from image diffusion to preserve temporal structure

Reference: “StableVideoSR: Video Super-Resolution via Stable Diffusion with Temporal Conditioning” (2024)

Constructors

Constructor Summary
StableVideoSR(NeuralNetworkArchitecture<T> architecture, StableVideoSROptions options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer, IConditioningModule<T> conditioner, UpscaleAVideoModel<T> diffusionCore) Creates a StableVideoSR model in native training mode.
StableVideoSR(NeuralNetworkArchitecture<T> architecture, string modelPath, StableVideoSROptions options) Creates a StableVideoSR model in ONNX inference mode.

Methods

Method Summary
ModelMetadata<T> GetModelMetadata()
Tensor<T> Upscale(Tensor<T> lowResFrames)
Tensor<T> UpscaleWithFlows(Tensor<T> lowResFrames, Tensor<T> forwardFlows, Tensor<T> backwardFlows) Upscales with externally estimated RAFT-compatible bidirectional flows in [B,2,F-1,H,W] layout, enabling the paper’s selected-step x0 propagation.
Vector<T> GetParameterGradients()