KlingModel<T>
Models & Types · AiDotNet.Diffusion.Video
Kling model — 3D spatiotemporal attention video generation by Kuaishou.
Inheritance: DiffusionModelBase<T> → LatentDiffusionModelBase<T> → VideoDiffusionModelBase<T> → KlingModel<T>
For Beginners
Kling is a video generation model from Kuaishou that creates high-quality videos from text or images.
How Kling works:
- Text prompt is encoded into a 4096-dimensional embedding
- A 3D DiT transformer generates video in compressed latent space
- Full 3D attention (not factorized) captures spatial AND temporal relationships
- A temporal VAE decompresses the latent video into pixel-space frames
Advantages:
- Very high video quality at 1080p resolution
- Long video support (up to 2 minutes)
- Strong physics and motion understanding
- Both text-to-video and image-to-video generation
Limitations:
- Very large model requiring significant GPU memory
- Proprietary architecture with limited public details
- Slower generation than smaller video models
How It Works
Kling uses a 3D VAE with full spatiotemporal attention for high-quality video generation with strong motion consistency and physics understanding. It supports up to 2 minutes of video at 1080p resolution.
Architecture components:
- DiT backbone with 3D full attention (36 layers, 2048 hidden dim)
- Temporal VAE for spatiotemporal compression (causal mode)
- Large-scale text encoder (4096-dim context)
- 16-channel latent space for high-fidelity reconstruction
Technical specifications:
- Architecture: DiT with full 3D spatiotemporal attention
- Parameters: ~5B+ (estimated)
- Backbone: 36 transformer layers, 2048 hidden dim, 16 attention heads
- Latent space: 16 channels with temporal VAE
- Text encoder: 4096-dimensional context embedding
- Max resolution: 1080p (1920x1080)
- Max duration: 2 minutes at 30 FPS
- Noise schedule: Flow matching
Reference: Kuaishou, “Kling: A Text-to-Video Generation Model”, 2024
Example
var referenceImage = Tensor<float>.CreateRandom(1, 3, 32, 32);
// Create with default settings
var kling = new KlingModel<float>();
// Generate video from text
var frames = kling.GenerateFromText(
prompt: "A drone flying over a mountain landscape at sunset",
numFrames: 60,
numInferenceSteps: 50,
guidanceScale: 7.0);
// Generate video from an image
var animatedFrames = kling.GenerateFromImage(
inputImage: referenceImage,
numFrames: 30);
Constructors
| Constructor | Summary |
|---|---|
KlingModel(NeuralNetworkArchitecture<T> architecture, DiffusionModelOptions<T> options, INoiseScheduler<T> scheduler, DiTNoisePredictor<T> dit, TemporalVAE<T> temporalVAE, IConditioningModule<T> conditioner, int defaultNumFrames, int defaultFPS, int? seed) |
Initializes a new instance of KlingModel with full customization support. |
Properties
| Property | Summary |
|---|---|
IConditioningModule<T> Conditioner { get; } |
|
INoisePredictor<T> NoisePredictor { get; } |
|
IVAEModel<T> TemporalVAE { get; } |
|
IVAEModel<T> VAE { get; } |
|
bool SupportsImageToVideo { get; } |
|
bool SupportsTextToVideo { get; } |
|
bool SupportsVideoToVideo { get; } |
|
int LatentChannels { get; } |
Methods
| Method | Summary |
|---|---|
ModelMetadata<T> GetModelMetadata() |