LTXVideoModel<T>
Models & Types · AiDotNet.Diffusion.Video
LTX-Video model for lightweight real-time video generation with extreme latent compression.
Inheritance: DiffusionModelBase<T> → LatentDiffusionModelBase<T> → VideoDiffusionModelBase<T> → LTXVideoModel<T>
For Beginners
LTX-Video generates videos faster than real-time by compressing heavily.
How LTX-Video works:
- Text prompt is encoded by T5-XXL into 4096-dim embeddings
- Video is compressed by the 3D causal VAE into 128-channel latent (192x compression)
- The lightweight DiT denoises the latent using flow matching
- The causal VAE decodes the latent back to 720p video
Key characteristics:
- ~2B parameters (lightweight for a video model)
- 192x spatiotemporal compression (highest in class)
- 128 latent channels for rich representations despite compression
- 720p, 5 seconds at 24 FPS, faster-than-real-time generation
- Open-source with Lightricks weights
When to use LTX-Video:
- Real-time or interactive video generation
- Applications requiring low latency
- Edge deployment with limited compute
- Rapid prototyping and iteration
Limitations:
- High compression may lose fine spatial details
- Quality may not match larger models (Sora, HunyuanVideo)
- Best for medium-resolution content
- Trade-off between speed and fidelity
How It Works
LTX-Video by Lightricks is designed for efficient video generation, using a lightweight DiT transformer operating in a highly compressed latent space via a 3D causal VAE. The extreme 192x compression ratio enables faster-than-real-time generation.
Architecture components:
- Lightweight DiT with 28 transformer layers and 1536 hidden dimension
- 16 attention heads for multi-head self-attention
- 3D causal VAE with 128 latent channels and 192x compression ratio
- T5-XXL text encoder for 4096-dim context embeddings
- Flow matching training objective for stable convergence
- Patch size 1 (operates on individual latent tokens)
Technical specifications:
- Architecture: Lightweight DiT with 3D causal VAE
- Hidden dimension: 1536
- Transformer layers: 28
- Attention heads: 16
- Patch size: 1 (direct latent token processing)
- Latent channels: 128 (extreme compression)
- Context dimension: 4096 (T5-XXL)
- VAE compression: 192x spatiotemporal
- Default: 121 frames at 24 FPS (~5 seconds)
- Training objective: Flow matching
- Total parameters: ~2B
Reference: Lightricks, “LTX-Video: Realtime Video Latent Diffusion”, 2024
Example
var referenceFrame = Tensor<float>.CreateRandom(1, 3, 32, 32);
// Create with industry-standard defaults
var ltxVideo = new LTXVideoModel<float>();
// Generate video from text (faster than real-time)
var video = ltxVideo.GenerateFromText(
prompt: "A time-lapse of clouds moving across a mountain range",
width: 1280,
height: 720,
numFrames: 121,
fps: 24,
numInferenceSteps: 30,
guidanceScale: 7.0);
// Generate video from an image
var animated = ltxVideo.GenerateFromImage(
inputImage: referenceFrame,
numFrames: 121,
numInferenceSteps: 30);
Constructors
| Constructor | Summary |
|---|---|
LTXVideoModel(NeuralNetworkArchitecture<T> architecture, DiffusionModelOptions<T> options, INoiseScheduler<T> scheduler, DiTNoisePredictor<T> dit, TemporalVAE<T> temporalVAE, IConditioningModule<T> conditioner, int defaultNumFrames, int defaultFPS, int? seed) |
Initializes a new instance of LTXVideoModel with full customization support. |
Properties
| Property | Summary |
|---|---|
IConditioningModule<T> Conditioner { get; } |
|
INoisePredictor<T> NoisePredictor { get; } |
|
IVAEModel<T> TemporalVAE { get; } |
|
IVAEModel<T> VAE { get; } |
|
bool SupportsImageToVideo { get; } |
|
bool SupportsTextToVideo { get; } |
|
bool SupportsVideoToVideo { get; } |
|
int LatentChannels { get; } |
Methods
| Method | Summary |
|---|---|
ModelMetadata<T> GetModelMetadata() |