VeoModel<T>
Models & Types · AiDotNet.Diffusion.Video
Veo model for Google’s high-fidelity cascaded video generation with temporal super-resolution.
Inheritance: DiffusionModelBase<T> → LatentDiffusionModelBase<T> → VideoDiffusionModelBase<T> → VeoModel<T>
For Beginners
Veo is Google’s top-tier video generation model.
How Veo works:
- Text is encoded by dual T5-XXL + CLIP encoders into 4096-dim embeddings
- Base model generates low-resolution video in compressed latent space
- Spatial super-resolution stage upscales each frame
- Temporal super-resolution stage adds interpolated frames
- The causal VAE decodes the final high-resolution video
Key characteristics:
- Cascaded architecture: base → spatial SR → temporal SR
- 1080p output with 60+ second duration capability
- Dual text encoding (T5-XXL + CLIP) for rich conditioning
- Veo 2 variant with improved quality and consistency
- 150 frames at 24 FPS by default (~6.25 seconds)
When to use Veo:
- Highest-quality video generation
- Long-duration video content
- 1080p high-resolution output
- Text-to-video, image-to-video, and video-to-video tasks
Limitations:
- Proprietary model (API-only access through Google)
- Very high compute requirements for cascaded generation
- Slower generation due to multi-stage pipeline
- Limited public information on exact architecture details
How It Works
Veo by Google DeepMind uses cascaded diffusion with temporal super-resolution for high-resolution, long-duration video generation. The base model generates at lower resolution, then spatial and temporal super-resolution stages produce the final output.
Architecture components:
- Cascaded DiT with 40 transformer layers and 2560 hidden dimension
- 20 attention heads with full spatiotemporal attention
- T5-XXL + CLIP dual text encoding for 4096-dim context
- 3D causal VAE with 16 latent channels and 3 temporal layers
- Cascaded pipeline: base → spatial SR → temporal SR
- Flow matching training objective
Technical specifications:
- Architecture: Cascaded DiT (base + spatial SR + temporal SR)
- Hidden dimension: 2560
- Transformer layers: 40
- Attention heads: 20
- Patch size: 2
- Latent channels: 16 (3D causal VAE)
- Context dimension: 4096 (T5-XXL + CLIP dual encoder)
- Output resolution: Up to 1080p
- Default: 150 frames at 24 FPS (~6.25 seconds)
- Veo 2: Enhanced quality variant (200 frames default)
- Training objective: Flow matching
Reference: Google DeepMind, “Veo: High-Fidelity Video Generation”, 2024
Example
var referenceFrame = Tensor<float>.CreateRandom(1, 3, 32, 32);
// Create Veo with defaults
var veo = new VeoModel<float>();
// Create Veo 2 variant
var veo2 = VeoModel<float>.CreateVeo2();
// Generate video from text
var video = veo.GenerateFromText(
prompt: "An aerial view of a coral reef teeming with tropical fish",
width: 1920,
height: 1080,
numFrames: 150,
fps: 24,
numInferenceSteps: 50,
guidanceScale: 7.5);
// Generate video from an image
var animated = veo.GenerateFromImage(
inputImage: referenceFrame,
numFrames: 150,
numInferenceSteps: 50);
Constructors
| Constructor | Summary |
|---|---|
VeoModel(NeuralNetworkArchitecture<T> architecture, DiffusionModelOptions<T> options, INoiseScheduler<T> scheduler, DiTNoisePredictor<T> dit, TemporalVAE<T> temporalVAE, IConditioningModule<T> conditioner, bool isVeo2, int defaultNumFrames, int defaultFPS, int? seed) |
Initializes a new instance of VeoModel with full customization support. |
Properties
| Property | Summary |
|---|---|
IConditioningModule<T> Conditioner { get; } |
|
INoisePredictor<T> NoisePredictor { get; } |
|
IVAEModel<T> TemporalVAE { get; } |
|
IVAEModel<T> VAE { get; } |
|
bool IsVeo2 { get; } |
Gets whether this is a Veo 2 variant with enhanced quality. |
bool SupportsImageToVideo { get; } |
|
bool SupportsTextToVideo { get; } |
|
bool SupportsVideoToVideo { get; } |
|
int LatentChannels { get; } |
Methods
| Method | Summary |
|---|---|
ModelMetadata<T> GetModelMetadata() |
|
VeoModel<T> CreateVeo2(IConditioningModule<T> conditioner) |
Creates a Veo 2 variant with enhanced quality and longer duration. |