Wonder3DModel<T>
Models & Types · AiDotNet.Diffusion.ThreeD
Wonder3D model for multi-view cross-domain diffusion with simultaneous RGB and normal map generation.
Inheritance: DiffusionModelBase<T> → LatentDiffusionModelBase<T> → ThreeDDiffusionModelBase<T> → Wonder3DModel<T>
For Beginners
Wonder3D creates both color views and normal maps for high-quality 3D reconstruction.
How Wonder3D works:
- Input image is encoded by CLIP into 768-dim features
- Shared U-Net backbone processes both RGB and normal map branches
- Cross-domain attention exchanges information between color and geometry branches
- 6 canonical views are generated simultaneously for both domains
- Normal maps provide geometric detail that color images alone cannot capture
- NeuS reconstruction combines both domains for high-quality textured meshes
Key characteristics:
- Dual-branch architecture: RGB images + normal maps
- Cross-domain attention ensures geometric consistency
- 6 canonical viewpoints for complete object coverage
- Normal maps improve reconstruction quality significantly
- SD 1.5 backbone with domain-specific adapters
- NeuS-based mesh reconstruction
When to use Wonder3D:
- High-quality single-image 3D reconstruction
- When geometric detail (from normal maps) is important
- When both texture and geometry quality matter
- Research on cross-domain multi-view generation
Limitations:
- Fixed 6-view output configuration
- Higher compute than single-branch methods (dual processing)
- Quality limited by SD 1.5 backbone resolution
- NeuS reconstruction can be slow for high-resolution meshes
How It Works
Wonder3D generates multi-view color images and normal maps simultaneously using cross-domain attention between RGB and normal map branches. A shared SD 1.5 backbone processes both domains with domain-specific adapters, producing 6 canonical views that are reconstructed into a textured 3D mesh via NeuS.
Architecture components:
- Dual-branch U-Net with shared SD 1.5 backbone (320 base channels, [1,2,4,4])
- Cross-domain attention between RGB and normal map branches
- CLIP image encoder for 768-dim conditioning
- 6 canonical viewpoints (front, back, left, right, top, bottom)
- NeuS reconstruction from cross-domain multi-view outputs
- Domain-specific adapters for RGB and normal map generation
Technical specifications:
- Architecture: Dual-branch U-Net with cross-domain attention
- Backbone: SD 1.5 (320 base channels, [1,2,4,4] multipliers)
- Cross-attention: 768-dim (CLIP)
- Output views: 6 (front, back, left, right, top, bottom)
- Output domains: RGB + normal maps
- Reconstruction: NeuS
- Default point count: 4,096
- Scheduler: DDIM with scaled linear beta
Reference: Long et al., “Wonder3D: Single Image to 3D using Cross-Domain Diffusion”, CVPR 2024
Example
var wonder3D = new Wonder3DModel<float>();
var mesh = wonder3D.GenerateMesh(
prompt: "A ceramic owl figurine",
resolution: 256,
numInferenceSteps: 50,
guidanceScale: 7.5);
Constructors
| Constructor | Summary |
|---|---|
Wonder3DModel(NeuralNetworkArchitecture<T> architecture, DiffusionModelOptions<T> options, INoiseScheduler<T> scheduler, UNetNoisePredictor<T> unet, StandardVAE<T> vae, IConditioningModule<T> conditioner, int defaultPointCount, int? seed) |
Properties
| Property | Summary |
|---|---|
IConditioningModule<T> Conditioner { get; } |
|
INoisePredictor<T> NoisePredictor { get; } |
|
IVAEModel<T> VAE { get; } |
|
bool SupportsMesh { get; } |
|
bool SupportsNovelView { get; } |
|
bool SupportsPointCloud { get; } |
|
bool SupportsScoreDistillation { get; } |
|
bool SupportsTexture { get; } |
|
int LatentChannels { get; } |
|
int NumViews { get; } |
Gets the number of canonical viewpoints generated simultaneously. |
Methods
| Method | Summary |
|---|---|
Mesh3D<T> GenerateMesh(string prompt, string negativePrompt, int resolution, int numInferenceSteps, double guidanceScale, int? seed) |
|
ModelMetadata<T> GetModelMetadata() |
|
Tensor<T> GeneratePointCloud(string prompt, string negativePrompt, int? numPoints, int numInferenceSteps, double guidanceScale, int? seed) |