StableVideoDiffusion<T>
Models & Types · AiDotNet.Diffusion.Video
Stable Video Diffusion (SVD) model for image-to-video generation.
Inheritance: DiffusionModelBase<T> → LatentDiffusionModelBase<T> → VideoDiffusionModelBase<T> → StableVideoDiffusion<T>
For Beginners
Think of SVD as “making a picture come to life.” You give it a single image, and it generates a short video showing how that scene might animate:
Example workflow:
- Input: Photo of a waterfall
- SVD analyzes the scene and understands what should move
- Output: 4-second video showing water flowing, mist rising
Key features:
- Image-to-video: Primary use case, animate still images
- Motion control: Adjust how much motion to add (motion bucket)
- Configurable length: Generate different numbers of frames
- High quality: Based on Stable Diffusion’s proven architecture
Compared to text-to-video:
- More predictable results (scene is defined by input image)
- Better quality (less ambiguity than text prompts)
- Faster generation (can use fewer denoising steps)
How It Works
Stable Video Diffusion generates short video clips from a single input image. It extends the Stable Diffusion architecture with temporal awareness, using a 3D U-Net for noise prediction and a temporal VAE for encoding/decoding.
Technical specifications:
- Default resolution: 576x1024 or 1024x576
- Default frames: 25 frames at 7 FPS (~3.5 seconds)
- Motion bucket ID: 1-255 (127 = moderate motion)
- Noise augmentation: 0.02 default for conditioning image
- Latent space: 4 channels, 8x spatial downsampling
Example
// Create a Stable Video Diffusion model
var svd = new StableVideoDiffusion<float>();
// Load your image (batch=1, channels=3, height=576, width=1024)
var inputImage = ImageHelper<float>.LoadImage("landscape.jpg");
// Generate video with default settings
var video = svd.GenerateFromImage(inputImage);
// Generate with custom motion (more movement)
var dynamicVideo = svd.GenerateFromImage(
inputImage,
numFrames: 25,
fps: 7,
motionBucketId: 200, // Higher = more motion
numInferenceSteps: 25,
seed: 42);
// Output shape: [1, 25, 3, 576, 1024] (batch, frames, channels, height, width)
// dynamicVideo is the result tensor — write it out with your own encoder.
Constructors
| Constructor | Summary |
|---|---|
StableVideoDiffusion(NeuralNetworkArchitecture<T> architecture, DiffusionModelOptions<T> options, INoiseScheduler<T> scheduler, VideoUNetPredictor<T> videoUNet, TemporalVAE<T> temporalVAE, IConditioningModule<T> conditioner, int defaultNumFrames, int defaultFPS, double noiseAugmentStrength) |
Initializes a new instance of StableVideoDiffusion with default parameters. |
Properties
| Property | Summary |
|---|---|
IConditioningModule<T> Conditioner { get; } |
Gets the conditioning module if available. |
INoisePredictor<T> NoisePredictor { get; } |
Gets the noise predictor used by this model. |
IVAEModel<T> TemporalVAE { get; } |
Gets the temporal VAE specifically for video operations. |
IVAEModel<T> VAE { get; } |
Gets the VAE used by this model for image encoding. |
VideoUNetPredictor<T> VideoUNet { get; } |
Gets the video U-Net predictor with image conditioning support. |
bool SupportsImageToVideo { get; } |
Gets whether this model supports image-to-video generation. |
bool SupportsTextToVideo { get; } |
Gets whether this model supports text-to-video generation. |
bool SupportsVideoToVideo { get; } |
Gets whether this model supports video-to-video transformation. |
int LatentChannels { get; } |
Gets the number of latent channels (4 for SVD). |
Methods
| Method | Summary |
|---|---|
Tensor<T> GenerateFromImage(Tensor<T> inputImage, int? numFrames, int? fps, int numInferenceSteps, int? motionBucketId, double noiseAugStrength, int? seed) |
Generates a video from an input image using image-to-video diffusion. |
Tensor<T> GenerateWithEndImageGuidance(Tensor<T> startImage, Tensor<T> endImage, int numFrames, int numInferenceSteps, int? seed) |
Generates video with motion guidance from a secondary image. |
Tensor<T> GenerateWithFirstFrame(Tensor<T> firstFrame, int numFrames, int motionBucketId, int numInferenceSteps, int? seed) |
Generates video with explicit first frame control. |
ValueTuple<int, int> GetRecommendedResolution(double aspectRatio) |
Gets the recommended resolution for SVD generation. |
Fields
| Field | Summary |
|---|---|
int DefaultHeight |
Default height for SVD generation. |
int DefaultWidth |
Default width for SVD generation. |