VideoDiffusionModelBase<T>
Base Classes · AiDotNet.Diffusion
Base class for video diffusion models that generate temporal sequences.
Inheritance: DiffusionModelBase<T> → LatentDiffusionModelBase<T> → VideoDiffusionModelBase<T>
Implements: IVideoDiffusionModel<T>
Known derived types: AllegroModel<T>, AnimateDiffModel<T>, CogVideoModel<T>, CogVideoX15Model<T>, CosmosModel<T>, DIAMONDModel<T>, DiffusionVideoSuperResolutionBase<T>, EmuVideo2Model<T>, EmuVideoModel<T>, FateZeroModel<T>, FlowVidModel<T>, FreeNoiseVideoModel<T>, GameGenXModel<T>, Genie2Model<T>, HunyuanVideo15Model<T>, HunyuanVideoModel<T>, InstructVid2VidModel<T>, Kling26Model<T>, KlingModel<T>, LTXVideoModel<T>, LatteModel<T>, LoongModel<T>, LumaRay2Model<T>, LumaRay3Model<T>, LumiereModel<T>, MAGI1Model<T>, MakeAVideoModel<T>, MinimaxVideoModel<T>, Mochi1Model<T>, Mochi1PreviewModel<T>, ModelScopeT2VModel<T>, MovieGenModel<T>, OasisModel<T>, OpenSora13Model<T>, OpenSora2Model<T>, OpenSoraModel<T>, Pika21Model<T>, PyramidFlowModel<T>, RunwayGen4Model<T>, RunwayGenModel<T>, Seedance1Model<T>, Show1Model<T>, SkyReelsV1Model<T>, SnapVideoModel<T>, Sora2Model<T>, SoraModel<T>, StableVideoDiffusion<T>, StepVideoModel<T>, StreamingT2VModel<T>, TokenFlowModel<T>, UniSimModel<T>, UpscaleAVideoModel<T>, Veo3Model<T>, VeoModel<T>, VideoCrafter2Model<T>, VideoCrafterModel<T>, VideoP2PModel<T>, VideoPoetModel<T>, Wan21Model<T>, Wan22Model<T> (+1 more)
For Beginners
This is the foundation for video generation models like Stable Video Diffusion and AnimateDiff. It extends latent diffusion to handle the temporal dimension, generating coherent video sequences where frames are consistent over time.
How It Works
This abstract base class provides common functionality for all video diffusion models, including image-to-video generation, text-to-video generation, video-to-video transformation, and frame interpolation.
Key capabilities:
- Image-to-Video: Animate a still image
- Text-to-Video: Generate video from text description
- Video-to-Video: Transform existing video style/content
- Frame interpolation: Increase frame rate smoothly
Properties
| Property | Summary |
|---|---|
IVAEModel<T> TemporalVAE { get; } |
Gets the temporal VAE for video encoding/decoding. |
bool SupportsImageToVideo { get; } |
|
bool SupportsTextToVideo { get; } |
|
bool SupportsVideoToVideo { get; } |
|
double NoiseAugStrength { get; set; } |
Gets the noise augmentation strength for input images. |
int DefaultFPS { get; } |
|
int DefaultNumFrames { get; } |
|
int MotionBucketId { get; } |
Methods
| Method | Summary |
|---|---|
Tensor<T> ExtractFrame(Tensor<T> video, int frameIndex) |
|
Tensor<T> FramesToVideo(Tensor<T>[] frames) |
|
Tensor<T> GenerateFromImage(Tensor<T> inputImage, int? numFrames, int? fps, int numInferenceSteps, int? motionBucketId, double noiseAugStrength, int? seed) |
|
Tensor<T> GenerateFromText(string prompt, string negativePrompt, int width, int height, int? numFrames, int? fps, int numInferenceSteps, double guidanceScale, int? seed) |
|
Tensor<T> InterpolateFrames(Tensor<T> video, int targetFPS, FrameInterpolationMethod interpolationMethod) |
|
Tensor<T> VideoToVideo(Tensor<T> inputVideo, string prompt, string negativePrompt, double strength, int numInferenceSteps, double guidanceScale, int? seed) |
|
void SetMotionBucketId(int bucketId) |