HunyuanVideoModel<T>
Models & Types · AiDotNet.Diffusion.Video
HunyuanVideo model for dual-stream DiT video generation with unified image-video capability.
Inheritance: DiffusionModelBase<T> → LatentDiffusionModelBase<T> → VideoDiffusionModelBase<T> → HunyuanVideoModel<T>
For Beginners
HunyuanVideo is Tencent’s open-source video generation model.
How HunyuanVideo works:
- Text prompt is encoded by an MLLM-based encoder into 4096-dim embeddings
- Video is compressed by the 3D causal VAE into a 16-channel latent (4x4x4 compression)
- The DS-DiT processes text and video in dual streams, then merges for joint attention
- Flow matching denoises the video latent over the scheduled timesteps
- The causal VAE decodes the latent back to 720p video
Key characteristics:
- 13B parameters total (one of the largest open-source video models)
- 720p resolution, 5+ second duration, 129 frames at 24 FPS
- Open-source weights available
- Unified image-video generation (both supported)
- 3D causal VAE enables temporal consistency
When to use HunyuanVideo:
- High-quality open-source video generation
- Text-to-video with strong prompt adherence
- Image-to-video animation
- Research and experimentation with large video models
Limitations:
- Very large model (13B parameters, requires significant GPU memory)
- Slower generation than lightweight models
- Causal VAE may have slight quality loss at clip boundaries
How It Works
HunyuanVideo by Tencent uses a “Dual-stream to Single-stream” Diffusion Transformer (DS-DiT) architecture with a 3D causal VAE for high-resolution video generation. The dual-stream design processes text and video tokens separately in early layers, then merges them for joint attention in later layers.
Architecture components:
- DS-DiT (Dual-stream to Single-stream DiT) with 40 transformer layers
- 3072 hidden dimension with 24 attention heads
- 3D causal VAE with 4x4x4 spatiotemporal compression (16 latent channels)
- MLLM-based text encoder for rich semantic understanding (4096-dim context)
- Flow matching training objective for stable convergence
Technical specifications:
- Architecture: DS-DiT (Dual-stream to Single-stream Diffusion Transformer)
- Hidden dimension: 3072
- Transformer layers: 40 (dual-stream early, single-stream late)
- Attention heads: 24
- Patch size: 2x2 spatiotemporal patches
- Latent channels: 16 (compressed by 3D causal VAE)
- Context dimension: 4096 (MLLM text encoder)
- VAE compression: 4x4x4 spatiotemporal
- Default: 129 frames at 24 FPS (~5.4 seconds)
- Training objective: Flow matching
- Total parameters: ~13B
Reference: Kong et al., “HunyuanVideo: A Systematic Framework For Large Video Generative Models”, 2024
Example
var referenceFrame = Tensor<float>.CreateRandom(1, 3, 32, 32);
// Create with industry-standard defaults
var hunyuan = new HunyuanVideoModel<float>();
// Generate video from text (requires conditioner)
var video = hunyuan.GenerateFromText(
prompt: "A golden retriever running through a sunlit meadow",
width: 1280,
height: 720,
numFrames: 129,
fps: 24,
numInferenceSteps: 50,
guidanceScale: 6.0);
// Generate video from an image
var animated = hunyuan.GenerateFromImage(
inputImage: referenceFrame,
numFrames: 129,
numInferenceSteps: 50);
Constructors
| Constructor | Summary |
|---|---|
HunyuanVideoModel(NeuralNetworkArchitecture<T> architecture, DiffusionModelOptions<T> options, INoiseScheduler<T> scheduler, DiTNoisePredictor<T> dit, TemporalVAE<T> temporalVAE, IConditioningModule<T> conditioner, int defaultNumFrames, int defaultFPS, int? seed) |
Initializes a new instance of HunyuanVideoModel with full customization support. |
Properties
| Property | Summary |
|---|---|
IConditioningModule<T> Conditioner { get; } |
|
INoisePredictor<T> NoisePredictor { get; } |
|
IVAEModel<T> TemporalVAE { get; } |
|
IVAEModel<T> VAE { get; } |
|
bool SupportsImageToVideo { get; } |
|
bool SupportsTextToVideo { get; } |
|
bool SupportsVideoToVideo { get; } |
|
int LatentChannels { get; } |
Methods
| Method | Summary |
|---|---|
ModelMetadata<T> GetModelMetadata() |