ThreeDDiffusionModelBase<T>

Base Classes · AiDotNet.Diffusion

Base class for 3D diffusion models that generate 3D content like point clouds, meshes, and scenes.

Inheritance: DiffusionModelBase<T> → LatentDiffusionModelBase<T> → ThreeDDiffusionModelBase<T>

Implements: I3DDiffusionModel<T>

Known derived types: DreamGaussianModel<T>, Instant3DModel<T>, LGMModel<T>, MVDreamModel<T>, Magic3DModel<T>, MeshyModel<T>, One2345Model<T>, PointEModel<T>, ShapEModel<T>, SyncDreamerModel<T>, TripoSRModel<T>, Wonder3DModel<T>

For Beginners

This is the foundation for 3D generation models like Point-E and Shap-E. It extends diffusion to create 3D objects instead of 2D images.

How It Works

This abstract base class provides common functionality for all 3D diffusion models, including point cloud generation, mesh generation, image-to-3D, novel view synthesis, and score distillation sampling.

Types of 3D generation:

  • Point Clouds: Sets of 3D points that form a shape
  • Meshes: Surfaces made of triangles (like in video games)
  • Textured Models: Meshes with colors and materials
  • Novel Views: New angles of an object from one image

How 3D diffusion works:

  1. Text-to-3D: Describe what you want and get a 3D model
  2. Image-to-3D: Turn a single photo into a full 3D model
  3. Score Distillation: Use 2D diffusion knowledge to guide 3D optimization

Properties

Property Summary
bool SupportsMesh { get; }
bool SupportsNovelView { get; }
bool SupportsPointCloud { get; }
bool SupportsScoreDistillation { get; }
bool SupportsTexture { get; }
double CoordinateScale { get; set; } Gets the coordinate scale for normalizing 3D positions.
int DefaultPointCount { get; }

Methods

Method Summary
Mesh3D<T> GenerateMesh(string prompt, string negativePrompt, int resolution, int numInferenceSteps, double guidanceScale, int? seed)
Mesh3D<T> ImageTo3D(Tensor<T> inputImage, int numViews, int numInferenceSteps, double guidanceScale, int? seed)
Mesh3D<T> PointCloudToMesh(Tensor<T> pointCloud, SurfaceReconstructionMethod method)
Tensor<T> ColorizePointCloud(Tensor<T> pointCloud, string prompt, int numInferenceSteps, int? seed)
Tensor<T> ComputeScoreDistillationGradients(Tensor<T> renderedViews, string prompt, int timestep, double guidanceScale)
Tensor<T> GeneratePointCloud(string prompt, string negativePrompt, int? numPoints, int numInferenceSteps, double guidanceScale, int? seed)
Tensor<T>[] SynthesizeNovelViews(Tensor<T> inputImage, ValueTuple<double, double>[] targetAngles, int numInferenceSteps, double guidanceScale, int? seed)