Zero123Model<T>

Models & Types · AiDotNet.Diffusion.ThreeD

Zero-1-to-3 model for novel view synthesis from a single image.

Inheritance: DiffusionModelBase<T> → LatentDiffusionModelBase<T> → Zero123Model<T>

For Beginners

Zero123 is like having a magical camera that can show you what an object looks like from different angles, even though you only gave it one photograph.

What it does:

  • Takes a single image of an object
  • Generates images of that same object from different viewpoints
  • Works with any object: cars, furniture, animals, etc.

Input parameters:

  • Image: The original photo of the object
  • Camera rotation: How much to rotate the view (polar/azimuth angles)
  • Scale change: How close/far to zoom

Use cases:

  • E-commerce: Show products from multiple angles
  • 3D reconstruction: Generate training data for 3D models
  • AR/VR: Create object previews from any angle
  • Game development: Generate sprite variations

How It Works

Zero-1-to-3 (Zero123) generates new viewpoints of an object from just a single input image. It uses camera pose conditioning to control the viewpoint change, enabling 3D-aware image generation without explicit 3D reconstruction.

Technical details:

  • Fine-tuned from Stable Diffusion
  • Uses CLIP image encoder for conditioning
  • Camera pose embedding via sinusoidal encoding
  • Supports arbitrary viewpoint changes
  • Can be used iteratively for 360° reconstruction

Reference: Liu et al., “Zero-1-to-3: Zero-shot One Image to 3D Object”, 2023

Example

// Create a Zero123 model
var zero123 = new Zero123Model<float>();
            
// Generate novel view
var inputImage = ImageHelper<float>.LoadImage("object.png");
var novelView = zero123.GenerateNovelView(
    inputImage: inputImage,
    polarAngle: 30.0,    // Rotate up 30 degrees
    azimuthAngle: 45.0,  // Rotate right 45 degrees
    radius: 1.0);        // Keep same distance
            
// Generate multiple views for 3D reconstruction
var views = zero123.Generate360Views(inputImage, numViews: 8);

Constructors

Constructor Summary
Zero123Model(NeuralNetworkArchitecture<T> architecture, DiffusionModelOptions<T> options, INoiseScheduler<T> scheduler, UNetNoisePredictor<T> unet, StandardVAE<T> vae, int imageSize, int? seed) Initializes a new instance of Zero123Model with full customization support.

Properties

Property Summary
IConditioningModule<T> Conditioner { get; }
INoisePredictor<T> NoisePredictor { get; }
IVAEModel<T> VAE { get; }
int LatentChannels { get; }

Methods

Method Summary
List<Tensor<T>> Generate360Views(Tensor<T> inputImage, int numViews, double polarAngle, int numInferenceSteps, double? guidanceScale, int? seed) Generates multiple views around an object (360° views).
List<Tensor<T>> GenerateMultipleViews(Tensor<T> inputImage, double[] azimuthAngles, double[] polarAngles, int numInferenceSteps, double? guidanceScale, int? seed) Generates views at multiple elevation angles.
Tensor<T> GenerateNovelView(Tensor<T> inputImage, double polarAngle, double azimuthAngle, double radius, int numInferenceSteps, double? guidanceScale, int? seed) Generates a novel view of an object.