SeedVR<T>
Models & Types · AiDotNet.Video.Enhancement
SeedVR: seeding infinity in diffusion transformer towards generic video restoration, as the authors released it.
Inheritance: NeuralNetworkBase<T> → VideoNeuralNetworkBase<T> → VideoSuperResolutionBase<T> → SeedVR<T>
For Beginners
SeedVR fixes degraded video (blur, noise, compression, low resolution) with a large generative model. It looks at the video in big overlapping tiles so it can handle any size, and uses a learned picture of what clean video looks like to fill in lost detail.
How It Works
SeedVR (Wang et al. 2025) restores video with a rectified-flow MM-DiT whose attention runs in large windows: each block cuts the latent token grid into windows (a fixed count at 720p, so any resolution works), every window’s video tokens attend to themselves and the prompt with a window-local 3D rotary embedding, and alternate blocks shift the windows by half. Video and text keep separate weights throughout. The low-quality clip enters as its causal-VAE latent, lightly noised and concatenated channel-wise with the noisy latent and a mask channel.
Sampling follows the release: 50 Euler steps over the uniform trailing timesteps with velocity (v = ε − x₀) prediction,
classifier-free guidance between the precomputed positive and negative prompt embeddings, and the clean estimate at the
last step. Training is SD3’s flow matching, which the paper follows: logit-normal timesteps shifted with the resolution
(the release’s timestep_transform), the MSE on the velocity, the condition noised at a quarter of the schedule.
The prompt embeddings are part of the release (text-encoder outputs computed once) and are held frozen; untrained, they
are zero. The input is upscaled bicubically by ScaleFactor in place of the release’s resize to
the 720p area, clips are padded to 4k + 1 frames and to multiples of 16 pixels, and the output is cropped back.
Constructors
| Constructor | Summary |
|---|---|
SeedVR(NeuralNetworkArchitecture<T> architecture, SeedVROptions options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer) |
Creates a SeedVR model in native training mode. |
SeedVR(NeuralNetworkArchitecture<T> architecture, string modelPath, SeedVROptions options) |
Creates a SeedVR model in native training mode. |
Methods
| Method | Summary |
|---|---|
LayerInputDomain GetInputDomain(int[] inputShape) |
Frames are pixels on the 0-255 scale; the network divides them by 255. |
LayerInputDomain GetOutputDomain(int[] outputShape) |
Restored frames come back on the same 0-255 scale. |
ModelMetadata<T> GetModelMetadata() |
|
Tensor<T> ForwardForTraining(Tensor<T> input) |
|
Tensor<T> Upscale(Tensor<T> lowResFrames) |