VideoCrafter2Model<T>
Models & Types · AiDotNet.Diffusion.Video
VideoCrafter 2 with improved quality and style fusion.
Inheritance: DiffusionModelBase<T> → LatentDiffusionModelBase<T> → VideoDiffusionModelBase<T> → VideoCrafter2Model<T>
For Beginners
VideoCrafter 2 improves on its predecessor with better visual quality and the ability to blend artistic styles into generated videos. It combines quality improvements with style transfer capabilities for creative video generation.
How It Works
References:
- Paper: “VideoCrafter2: Overcoming Data Limitations for High-Quality Video Diffusion Models” (Tencent, 2024)
VideoCrafter 2 improves video quality by overcoming data limitations through better data curation and augmentation strategies. Uses a Video LDM architecture with factorized spatiotemporal attention and CLIP text encoding. Achieves significant quality improvements over VideoCrafter 1 with better temporal consistency and visual fidelity.
Technical specifications:
- Architecture: Video LDM + Factorized Spatiotemporal Attention + CLIP
- Latent channels: 4
- Default: 16 frames at 8 FPS
- Supports I2V: Yes | T2V: Yes | V2V: No
Example
var options = new DiffusionModelOptions<float> { LatentChannels = 4, DefaultInferenceSteps = 50 };
var noise = Tensor<float>.CreateRandom(new[] { 1, 4, 16, 40, 64 });
var trainX = Tensor<float>.CreateRandom(4, 8);
var trainY = Tensor<float>.CreateRandom(4, 2);
var result = new AiModelBuilder<float, Tensor<float>, Tensor<float>>()
.ConfigureModel(new VideoCrafter2Model<float>(options: options))
.Build(trainX, trainY);
var video = result.Predict(noise);
Constructors
| Constructor | Summary |
|---|---|
VideoCrafter2Model(NeuralNetworkArchitecture<T> architecture, DiffusionModelOptions<T> options, INoiseScheduler<T> scheduler, VideoUNetPredictor<T> predictor, TemporalVAE<T> temporalVAE, IConditioningModule<T> conditioner, int defaultNumFrames, int defaultFPS) |
Initializes a new instance of VideoCrafter2Model with full customization support. |
Properties
| Property | Summary |
|---|---|
IConditioningModule<T> Conditioner { get; } |
|
INoisePredictor<T> NoisePredictor { get; } |
|
IVAEModel<T> TemporalVAE { get; } |
|
IVAEModel<T> VAE { get; } |
|
bool SupportsImageToVideo { get; } |
|
bool SupportsTextToVideo { get; } |
|
bool SupportsVideoToVideo { get; } |
|
int LatentChannels { get; } |
Methods
| Method | Summary |
|---|---|
ModelMetadata<T> GetModelMetadata() |