EmuVideoModel<T>

Models & Types · AiDotNet.Diffusion.Video.AudioVisual

Emu Video high-quality video generation with temporal consistency.

Inheritance: DiffusionModelBase<T> → LatentDiffusionModelBase<T> → VideoDiffusionModelBase<T> → EmuVideoModel<T>

For Beginners

Emu Video from Meta generates high-quality videos with strong temporal consistency, meaning objects maintain their appearance and motion stays smooth throughout the clip. It uses a factored approach separating image generation from temporal animation.

How It Works

References:

  • Paper: “Emu Video: Factorizing Text-to-Video Generation by Explicit Image Conditioning” (Meta, 2023)

Emu Video generates high-quality videos by factorizing the text-to-video problem into two stages: first generating a conditioning image from text, then animating it. This factorization simplifies the learning problem and produces videos with strong temporal consistency and natural motion.

Technical specifications:

  • Architecture: Factorized T2V: Text-to-Image + Image-to-Video
  • Latent channels: 16
  • Default: 16 frames at 16 FPS
  • Supports I2V: Yes | T2V: Yes | V2V: No

Example

// Create Emu Video for text-to-video generation
var model = new EmuVideoModel<float>();
            
// Generate video from text with factorized approach
var video = model.GenerateFromText(
    prompt: "A serene lake at sunset with gentle ripples",
    width: 512, height: 512,
    numFrames: 16, fps: 16);

Constructors

Constructor Summary
EmuVideoModel(NeuralNetworkArchitecture<T> architecture, DiffusionModelOptions<T> options, INoiseScheduler<T> scheduler, DiTNoisePredictor<T> predictor, TemporalVAE<T> temporalVAE, IConditioningModule<T> conditioner, int defaultNumFrames, int defaultFPS, int? seed) Initializes a new instance of EmuVideoModel with full customization support.

Properties

Property Summary
IConditioningModule<T> Conditioner { get; }
INoisePredictor<T> NoisePredictor { get; }
IVAEModel<T> TemporalVAE { get; }
IVAEModel<T> VAE { get; }
bool SupportsImageToVideo { get; }
bool SupportsTextToVideo { get; }
bool SupportsVideoToVideo { get; }
int LatentChannels { get; }

Methods

Method Summary
ModelMetadata<T> GetModelMetadata()