Genie2Model<T>
Models & Types · AiDotNet.Diffusion.Video.WorldModels
Genie 2 real-time interactive 3D environment generation.
Inheritance: DiffusionModelBase<T> → LatentDiffusionModelBase<T> → VideoDiffusionModelBase<T> → Genie2Model<T>
For Beginners
Genie 2 from Google DeepMind generates real-time interactive 3D environments from a single image. It creates explorable worlds where user actions produce consistent, realistic responses.
How It Works
References:
- Reference: Google DeepMind Genie 2 (2024)
Genie 2 generates real-time interactive 3D environments from single images or text descriptions. The model enables interactive exploration of generated worlds with consistent physics and persistent state. Users can navigate and interact with the generated environment in real-time.
Technical specifications:
- Architecture: Action-Conditioned DiT + Interactive 3D + Real-Time
- Latent channels: 16
- Default: 60 frames at 30 FPS
- Supports I2V: Yes | T2V: Yes | V2V: No
Example
var options = new DiffusionModelOptions<float> { LatentChannels = 16, DefaultInferenceSteps = 10 };
var actionInput = Tensor<float>.CreateRandom(new[] { 1, 16, 60, 90, 160 });
var trainX = Tensor<float>.CreateRandom(4, 8);
var trainY = Tensor<float>.CreateRandom(4, 2);
var result = new AiModelBuilder<float, Tensor<float>, Tensor<float>>()
.ConfigureModel(new Genie2Model<float>(options: options))
.Build(trainX, trainY);
var world = result.Predict(actionInput);
Constructors
| Constructor | Summary |
|---|---|
Genie2Model(NeuralNetworkArchitecture<T> architecture, DiffusionModelOptions<T> options, INoiseScheduler<T> scheduler, DiTNoisePredictor<T> predictor, TemporalVAE<T> temporalVAE, IConditioningModule<T> conditioner, int defaultNumFrames, int defaultFPS) |
Initializes a new instance of Genie2Model with full customization support. |
Properties
| Property | Summary |
|---|---|
IConditioningModule<T> Conditioner { get; } |
|
INoisePredictor<T> NoisePredictor { get; } |
|
IVAEModel<T> TemporalVAE { get; } |
|
IVAEModel<T> VAE { get; } |
|
bool SupportsImageToVideo { get; } |
|
bool SupportsTextToVideo { get; } |
|
bool SupportsVideoToVideo { get; } |
|
int LatentChannels { get; } |
Methods
| Method | Summary |
|---|---|
ModelMetadata<T> GetModelMetadata() |