MVDreamModel<T>
Models & Types · AiDotNet.Diffusion.ThreeD
MVDream - Multi-View Diffusion Model for 3D-consistent image generation.
Inheritance: DiffusionModelBase<T> → LatentDiffusionModelBase<T> → ThreeDDiffusionModelBase<T> → MVDreamModel<T>
For Beginners
MVDream creates multiple images of the same object from different angles, all consistent with each other:
Example: “A red sports car”
- Front view: shows the car’s front grille and headlights
- Side view: shows the profile with wheels and doors
- Back view: shows tail lights and rear design
- All views are consistent (same car, same color, same features)
This consistency is what enables 3D reconstruction:
- Multiple views + triangulation = 3D model
- Can be used with Score Distillation for high-quality 3D
How It Works
MVDream is a multi-view diffusion model that generates 3D-consistent images from multiple viewpoints simultaneously. It enables high-quality 3D generation by leveraging multi-view supervision during training.
Key capabilities:
- Multi-View Generation: Generate multiple consistent views of an object
- Text-to-3D: Create 3D content from text descriptions
- Image-to-3D: Convert single images to 3D representations
- Score Distillation Sampling (SDS): Guide NeRF/3DGS optimization
- Novel View Synthesis: Generate unseen viewpoints of objects
Technical specifications:
- Image resolution: 256x256 per view (default)
- Number of views: 4 (orthogonal) or 8 (comprehensive)
- Latent channels: 4 (Stable Diffusion compatible)
- Context dimension: 1024 (CLIP/T5 embeddings)
- Camera model: Spherical coordinates (azimuth, elevation, radius)
Example
var nerFRenderOutput = Tensor<float>.CreateRandom(1, 3, 32, 32);
// Create MVDream model
var mvdream = new MVDreamModel<float>();
// Generate 4 views of an object
var views = mvdream.GenerateMultiView(
prompt: "A cute robot toy",
numViews: 4,
numInferenceSteps: 50,
guidanceScale: 7.5);
// Generate for score distillation (SDS)
var sdsGradient = mvdream.ComputeScoreDistillationGradients(
renderedViews: nerFRenderOutput,
prompt: "A cute robot toy",
timestep: 500,
guidanceScale: 100.0);
Constructors
| Constructor | Summary |
|---|---|
MVDreamModel(NeuralNetworkArchitecture<T> architecture, DiffusionModelOptions<T> options, INoiseScheduler<T> scheduler, MultiViewUNet<T> multiViewUNet, StandardVAE<T> imageVAE, IConditioningModule<T> textConditioner, IConditioningModule<T> imageConditioner, MVDreamConfig config, int? seed) |
Initializes a new MVDream model with full customization support. |
Properties
| Property | Summary |
|---|---|
CameraEmbedding<T> CameraEmbedding { get; } |
Gets the camera embedding module. |
IConditioningModule<T> Conditioner { get; } |
|
IConditioningModule<T> ImageConditioner { get; } |
Gets the image conditioner for image-to-3D tasks. |
INoisePredictor<T> NoisePredictor { get; } |
|
IVAEModel<T> VAE { get; } |
|
MVDreamConfig Config { get; } |
Gets the model configuration. |
bool SupportsMesh { get; } |
|
bool SupportsNovelView { get; } |
|
bool SupportsPointCloud { get; } |
|
bool SupportsScoreDistillation { get; } |
|
bool SupportsTexture { get; } |
|
int LatentChannels { get; } |
Methods
| Method | Summary |
|---|---|
Mesh3D<T> ImageTo3D(Tensor<T> inputImage, int numViews, int numInferenceSteps, double guidanceScale, int? seed) |
Generates 3D from a single input image. |
Tensor<T> ComputeScoreDistillationGradients(Tensor<T> renderedViews, string prompt, int timestep, double guidanceScale) |
Computes Score Distillation Sampling gradients for 3D optimization. |
Tensor<T>[] GenerateMultiView(string prompt, string negativePrompt, int numViews, int numInferenceSteps, double guidanceScale, double elevation, int? seed) |
Generates multiple consistent views from a text prompt. |
Tensor<T>[] GenerateNovelViewsFromImage(Tensor<T> inputImage, int numViews, int numInferenceSteps, double guidanceScale, int? seed) |
Generates novel views from a single input image. |
Fields
| Field | Summary |
|---|---|
double DEFAULT_CAMERA_DISTANCE |
Default camera distance from object. |
int MVDREAM_BASE_CHANNELS |
MVDream base channels for U-Net. |
int MVDREAM_CONTEXT_DIM |
Context dimension for text/image conditioning. |
int MVDREAM_IMAGE_SIZE |
Default image resolution for each view. |
int MVDREAM_LATENT_CHANNELS |
MVDream latent channels (Stable Diffusion compatible). |