Wonder3DModel<T>

Models & Types · AiDotNet.Diffusion.ThreeD

Wonder3D model for multi-view cross-domain diffusion with simultaneous RGB and normal map generation.

Inheritance: DiffusionModelBase<T> → LatentDiffusionModelBase<T> → ThreeDDiffusionModelBase<T> → Wonder3DModel<T>

For Beginners

Wonder3D creates both color views and normal maps for high-quality 3D reconstruction.

How Wonder3D works:

  1. Input image is encoded by CLIP into 768-dim features
  2. Shared U-Net backbone processes both RGB and normal map branches
  3. Cross-domain attention exchanges information between color and geometry branches
  4. 6 canonical views are generated simultaneously for both domains
  5. Normal maps provide geometric detail that color images alone cannot capture
  6. NeuS reconstruction combines both domains for high-quality textured meshes

Key characteristics:

  • Dual-branch architecture: RGB images + normal maps
  • Cross-domain attention ensures geometric consistency
  • 6 canonical viewpoints for complete object coverage
  • Normal maps improve reconstruction quality significantly
  • SD 1.5 backbone with domain-specific adapters
  • NeuS-based mesh reconstruction

When to use Wonder3D:

  • High-quality single-image 3D reconstruction
  • When geometric detail (from normal maps) is important
  • When both texture and geometry quality matter
  • Research on cross-domain multi-view generation

Limitations:

  • Fixed 6-view output configuration
  • Higher compute than single-branch methods (dual processing)
  • Quality limited by SD 1.5 backbone resolution
  • NeuS reconstruction can be slow for high-resolution meshes

How It Works

Wonder3D generates multi-view color images and normal maps simultaneously using cross-domain attention between RGB and normal map branches. A shared SD 1.5 backbone processes both domains with domain-specific adapters, producing 6 canonical views that are reconstructed into a textured 3D mesh via NeuS.

Architecture components:

  • Dual-branch U-Net with shared SD 1.5 backbone (320 base channels, [1,2,4,4])
  • Cross-domain attention between RGB and normal map branches
  • CLIP image encoder for 768-dim conditioning
  • 6 canonical viewpoints (front, back, left, right, top, bottom)
  • NeuS reconstruction from cross-domain multi-view outputs
  • Domain-specific adapters for RGB and normal map generation

Technical specifications:

  • Architecture: Dual-branch U-Net with cross-domain attention
  • Backbone: SD 1.5 (320 base channels, [1,2,4,4] multipliers)
  • Cross-attention: 768-dim (CLIP)
  • Output views: 6 (front, back, left, right, top, bottom)
  • Output domains: RGB + normal maps
  • Reconstruction: NeuS
  • Default point count: 4,096
  • Scheduler: DDIM with scaled linear beta

Reference: Long et al., “Wonder3D: Single Image to 3D using Cross-Domain Diffusion”, CVPR 2024

Example

var wonder3D = new Wonder3DModel<float>();
var mesh = wonder3D.GenerateMesh(
    prompt: "A ceramic owl figurine",
    resolution: 256,
    numInferenceSteps: 50,
    guidanceScale: 7.5);

Constructors

Constructor Summary
Wonder3DModel(NeuralNetworkArchitecture<T> architecture, DiffusionModelOptions<T> options, INoiseScheduler<T> scheduler, UNetNoisePredictor<T> unet, StandardVAE<T> vae, IConditioningModule<T> conditioner, int defaultPointCount, int? seed)

Properties

Property Summary
IConditioningModule<T> Conditioner { get; }
INoisePredictor<T> NoisePredictor { get; }
IVAEModel<T> VAE { get; }
bool SupportsMesh { get; }
bool SupportsNovelView { get; }
bool SupportsPointCloud { get; }
bool SupportsScoreDistillation { get; }
bool SupportsTexture { get; }
int LatentChannels { get; }
int NumViews { get; } Gets the number of canonical viewpoints generated simultaneously.

Methods

Method Summary
Mesh3D<T> GenerateMesh(string prompt, string negativePrompt, int resolution, int numInferenceSteps, double guidanceScale, int? seed)
ModelMetadata<T> GetModelMetadata()
Tensor<T> GeneratePointCloud(string prompt, string negativePrompt, int? numPoints, int numInferenceSteps, double guidanceScale, int? seed)