CogVideoX15Model<T>
Models & Types · AiDotNet.Diffusion.Video
CogVideoX 1.5 model with 10-second any-resolution video generation.
Inheritance: DiffusionModelBase<T> → LatentDiffusionModelBase<T> → VideoDiffusionModelBase<T> → CogVideoX15Model<T>
For Beginners
CogVideoX 1.5 creates 10-second videos at any resolution from text prompts. It compresses video efficiently using a 3D causal VAE (reducing memory 128x) and uses specialized expert transformer blocks that understand both spatial layout and temporal motion.
How It Works
References:
- Paper: “CogVideoX: Text-to-Video Diffusion Models with An Expert Transformer” (Zhipu AI/THUDM, 2024)
CogVideoX 1.5 extends the original CogVideoX to 10-second generation with any-resolution support. Uses 3D causal VAE with 4x temporal and 8x spatial compression, expert transformer blocks with adaptive layer normalization, and T5 text encoding. The 5B parameter model supports both text-to-video and image-to-video generation at up to 720p.
Technical specifications:
- Architecture: 3D Causal VAE + Expert Transformer + T5
- Latent channels: 16
- Default: 80 frames at 8 FPS
- Supports I2V: Yes | T2V: Yes | V2V: No
Example
var options = new DiffusionModelOptions<float> { LatentChannels = 16, DefaultInferenceSteps = 50 };
var noise = Tensor<float>.CreateRandom(new[] { 1, 16, 80, 60, 90 });
var trainX = Tensor<float>.CreateRandom(4, 8);
var trainY = Tensor<float>.CreateRandom(4, 2);
var result = new AiModelBuilder<float, Tensor<float>, Tensor<float>>()
.ConfigureModel(new CogVideoX15Model<float>(options: options))
.Build(trainX, trainY);
var video = result.Predict(noise);
Constructors
| Constructor | Summary |
|---|---|
CogVideoX15Model(NeuralNetworkArchitecture<T> architecture, DiffusionModelOptions<T> options, INoiseScheduler<T> scheduler, DiTNoisePredictor<T> predictor, TemporalVAE<T> temporalVAE, IConditioningModule<T> conditioner, int defaultNumFrames, int defaultFPS, int? seed) |
Initializes a new instance of CogVideoX15Model with full customization support. |
Properties
| Property | Summary |
|---|---|
IConditioningModule<T> Conditioner { get; } |
|
INoisePredictor<T> NoisePredictor { get; } |
|
IVAEModel<T> TemporalVAE { get; } |
|
IVAEModel<T> VAE { get; } |
|
bool SupportsImageToVideo { get; } |
|
bool SupportsTextToVideo { get; } |
|
bool SupportsVideoToVideo { get; } |
|
int LatentChannels { get; } |
Methods
| Method | Summary |
|---|---|
ModelMetadata<T> GetModelMetadata() |