Genie2Model<T>

Models & Types · AiDotNet.Diffusion.Video.WorldModels

Genie 2 real-time interactive 3D environment generation.

Inheritance: DiffusionModelBase<T> → LatentDiffusionModelBase<T> → VideoDiffusionModelBase<T> → Genie2Model<T>

For Beginners

Genie 2 from Google DeepMind generates real-time interactive 3D environments from a single image. It creates explorable worlds where user actions produce consistent, realistic responses.

How It Works

References:

  • Reference: Google DeepMind Genie 2 (2024)

Genie 2 generates real-time interactive 3D environments from single images or text descriptions. The model enables interactive exploration of generated worlds with consistent physics and persistent state. Users can navigate and interact with the generated environment in real-time.

Technical specifications:

  • Architecture: Action-Conditioned DiT + Interactive 3D + Real-Time
  • Latent channels: 16
  • Default: 60 frames at 30 FPS
  • Supports I2V: Yes | T2V: Yes | V2V: No

Example

var options = new DiffusionModelOptions<float> { LatentChannels = 16, DefaultInferenceSteps = 10 };
var actionInput = Tensor<float>.CreateRandom(new[] { 1, 16, 60, 90, 160 });
var trainX = Tensor<float>.CreateRandom(4, 8);
var trainY = Tensor<float>.CreateRandom(4, 2);
var result = new AiModelBuilder<float, Tensor<float>, Tensor<float>>()
    .ConfigureModel(new Genie2Model<float>(options: options))
    .Build(trainX, trainY);
var world = result.Predict(actionInput);

Constructors

Constructor Summary
Genie2Model(NeuralNetworkArchitecture<T> architecture, DiffusionModelOptions<T> options, INoiseScheduler<T> scheduler, DiTNoisePredictor<T> predictor, TemporalVAE<T> temporalVAE, IConditioningModule<T> conditioner, int defaultNumFrames, int defaultFPS) Initializes a new instance of Genie2Model with full customization support.

Properties

Property Summary
IConditioningModule<T> Conditioner { get; }
INoisePredictor<T> NoisePredictor { get; }
IVAEModel<T> TemporalVAE { get; }
IVAEModel<T> VAE { get; }
bool SupportsImageToVideo { get; }
bool SupportsTextToVideo { get; }
bool SupportsVideoToVideo { get; }
int LatentChannels { get; }

Methods

Method Summary
ModelMetadata<T> GetModelMetadata()