VideoCrafter2Model<T>

Models & Types · AiDotNet.Diffusion.Video

VideoCrafter 2 with improved quality and style fusion.

Inheritance: DiffusionModelBase<T> → LatentDiffusionModelBase<T> → VideoDiffusionModelBase<T> → VideoCrafter2Model<T>

For Beginners

VideoCrafter 2 improves on its predecessor with better visual quality and the ability to blend artistic styles into generated videos. It combines quality improvements with style transfer capabilities for creative video generation.

How It Works

References:

  • Paper: “VideoCrafter2: Overcoming Data Limitations for High-Quality Video Diffusion Models” (Tencent, 2024)

VideoCrafter 2 improves video quality by overcoming data limitations through better data curation and augmentation strategies. Uses a Video LDM architecture with factorized spatiotemporal attention and CLIP text encoding. Achieves significant quality improvements over VideoCrafter 1 with better temporal consistency and visual fidelity.

Technical specifications:

  • Architecture: Video LDM + Factorized Spatiotemporal Attention + CLIP
  • Latent channels: 4
  • Default: 16 frames at 8 FPS
  • Supports I2V: Yes | T2V: Yes | V2V: No

Example

var options = new DiffusionModelOptions<float> { LatentChannels = 4, DefaultInferenceSteps = 50 };
var noise = Tensor<float>.CreateRandom(new[] { 1, 4, 16, 40, 64 });
var trainX = Tensor<float>.CreateRandom(4, 8);
var trainY = Tensor<float>.CreateRandom(4, 2);
var result = new AiModelBuilder<float, Tensor<float>, Tensor<float>>()
    .ConfigureModel(new VideoCrafter2Model<float>(options: options))
    .Build(trainX, trainY);
var video = result.Predict(noise);

Constructors

Constructor Summary
VideoCrafter2Model(NeuralNetworkArchitecture<T> architecture, DiffusionModelOptions<T> options, INoiseScheduler<T> scheduler, VideoUNetPredictor<T> predictor, TemporalVAE<T> temporalVAE, IConditioningModule<T> conditioner, int defaultNumFrames, int defaultFPS) Initializes a new instance of VideoCrafter2Model with full customization support.

Properties

Property Summary
IConditioningModule<T> Conditioner { get; }
INoisePredictor<T> NoisePredictor { get; }
IVAEModel<T> TemporalVAE { get; }
IVAEModel<T> VAE { get; }
bool SupportsImageToVideo { get; }
bool SupportsTextToVideo { get; }
bool SupportsVideoToVideo { get; }
int LatentChannels { get; }

Methods

Method Summary
ModelMetadata<T> GetModelMetadata()