StableVideoSR<T>
Models & Types · AiDotNet.Video.Enhancement
StableVideoSR: stable diffusion with temporal conditioning for video super-resolution.
Inheritance: NeuralNetworkBase<T> → VideoNeuralNetworkBase<T> → VideoSuperResolutionBase<T> → StableVideoSR<T>
For Beginners
StableVideoSR extends the popular Stable Diffusion image AI to handle video. It adds “temporal awareness” so the model considers what happened in previous and next frames when upscaling each frame, preventing the flickering that occurs when frames are processed independently.
Usage:
How It Works
StableVideoSR (2024) adapts the Stable Diffusion architecture for video SR:
- Temporal conditioning modules: cross-attention layers inserted between spatial attention
in the U-Net attend to features from adjacent frames, maintaining temporal coherence
- Direct low-resolution conditioning: noised RGB frames are concatenated with the latent
input, matching the released seven-channel U-Net contract
- Classifier-free guidance: balances between faithful reconstruction and generative
enhancement during inference
- Noise schedule: adapted from image diffusion to preserve temporal structure
Reference: “StableVideoSR: Video Super-Resolution via Stable Diffusion with Temporal Conditioning” (2024)
Constructors
| Constructor | Summary |
|---|---|
StableVideoSR(NeuralNetworkArchitecture<T> architecture, StableVideoSROptions options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer, IConditioningModule<T> conditioner, UpscaleAVideoModel<T> diffusionCore) |
Creates a StableVideoSR model in native training mode. |
StableVideoSR(NeuralNetworkArchitecture<T> architecture, string modelPath, StableVideoSROptions options) |
Creates a StableVideoSR model in ONNX inference mode. |
Methods
| Method | Summary |
|---|---|
ModelMetadata<T> GetModelMetadata() |
|
Tensor<T> Upscale(Tensor<T> lowResFrames) |
|
Tensor<T> UpscaleWithFlows(Tensor<T> lowResFrames, Tensor<T> forwardFlows, Tensor<T> backwardFlows) |
Upscales with externally estimated RAFT-compatible bidirectional flows in [B,2,F-1,H,W] layout, enabling the paper’s selected-step x0 propagation. |
Vector<T> GetParameterGradients() |