ShapEModel<T>
Models & Types · AiDotNet.Diffusion.ThreeD
Shap-E model for text-to-3D and image-to-3D generation with implicit neural representations.
Inheritance: DiffusionModelBase<T> → LatentDiffusionModelBase<T> → ThreeDDiffusionModelBase<T> → ShapEModel<T>
For Beginners
Shap-E creates 3D objects that you can view from any angle:
What is an Implicit Neural Representation (NeRF)?
- A neural network that knows the 3D shape
- Input: 3D coordinates (x, y, z)
- Output: Color and density at that point
- Can render views from ANY angle without artifacts
Comparison with Point-E:
| Feature | Point-E | Shap-E |
|---|---|---|
| Output | Point cloud | Neural field |
| Quality | Good | Better |
| Rendering | Fast | Slower |
| Mesh export | Reconstruction | Direct SDF |
| Memory | Lower | Higher |
Example: “A red chair”
- Shap-E generates network weights (latent representation)
- These weights define a neural network
- Query (x,y,z) -> neural network -> color, density
- Render from any view or extract mesh via marching cubes
Use cases:
- High-quality 3D assets
- Novel view synthesis
- Direct mesh export with SDF
- View-consistent 3D models
How It Works
Shap-E is OpenAI’s model for generating 3D objects as implicit neural representations (NeRFs). Unlike Point-E which generates point clouds, Shap-E generates parameters for a neural network that represents the 3D shape, which can then be rendered from any angle or converted to meshes.
Technical specifications:
- Latent dimension: 1024 parameters per shape
- Output: NeRF weights or SDF (Signed Distance Function)
- Rendering: Differentiable volumetric rendering
- Mesh export: Marching cubes on SDF
- Inference: ~64 steps
Example
// Create a Shap-E model
var shapE = new ShapEModel<float>();
// Generate a 3D shape from text
var latent = shapE.GenerateLatent(
prompt: "A wooden chair",
numInferenceSteps: 64,
guidanceScale: 15.0);
// Render from a specific view
var image = shapE.RenderView(latent, cameraPosition: (0, 0, 2), lookAt: (0, 0, 0));
// Export to mesh
var (vertices, faces) = shapE.ExtractMesh(latent, resolution: 64);
// Or use the high-level API
var mesh = shapE.GenerateMesh(
prompt: "A red sports car",
resolution: 128);
// mesh is the result tensor — write it out with your own encoder.
Constructors
| Constructor | Summary |
|---|---|
ShapEModel(NeuralNetworkArchitecture<T> architecture, DiffusionModelOptions<T> options, INoiseScheduler<T> scheduler, DiTNoisePredictor<T> latentPredictor, IConditioningModule<T> conditioner, bool useSDFMode, int defaultPointCount, int? seed) |
Initializes a new Shap-E model with full customization support. |
Properties
| Property | Summary |
|---|---|
IConditioningModule<T> Conditioner { get; } |
|
INoisePredictor<T> NoisePredictor { get; } |
|
IVAEModel<T> VAE { get; } |
|
bool SupportsMesh { get; } |
|
bool SupportsNovelView { get; } |
|
bool SupportsPointCloud { get; } |
|
bool SupportsScoreDistillation { get; } |
|
bool SupportsTexture { get; } |
|
bool UseSDFMode { get; } |
Gets whether this model uses SDF mode. |
int LatentChannels { get; } |
|
int LatentDimension { get; } |
Gets the latent dimension. |
Methods
| Method | Summary |
|---|---|
Mesh3D<T> GenerateMesh(string prompt, string negativePrompt, int resolution, int numInferenceSteps, double guidanceScale, int? seed) |
Generates a mesh directly from a text prompt. |
Mesh3D<T> ImageTo3D(Tensor<T> inputImage, int numViews, int numInferenceSteps, double guidanceScale, int? seed) |
Image-to-3D with Shap-E’s image-conditioned pipeline (Jun and Nichol, 2023, image300M): the image conditions the latent diffusion (GenerateLatentFromImage), and the latent’s implicit SDF is decoded and its zero level set extracted on a 64… |
ModelMetadata<T> GetModelMetadata() |
|
Tensor<T> GenerateLatent(string prompt, string negativePrompt, int numInferenceSteps, double guidanceScale, int? seed) |
Generates a latent representation of a 3D shape from text. |
Tensor<T> GenerateLatentFromImage(Tensor<T> image, int numInferenceSteps, double guidanceScale, int? seed) |
Generates a latent from an image. |
Tensor<T> GeneratePointCloud(string prompt, string negativePrompt, int? numPoints, int numInferenceSteps, double guidanceScale, int? seed) |
|
Tensor<T> RenderView(Tensor<T> latent, ValueTuple<double, double, double> cameraPosition, ValueTuple<double, double, double> lookAt, int imageSize, int numSamples) |
Renders a view of the shape from a camera position. |
Tensor<T> SamplePointCloud(Tensor<T> latent, int? numPoints, int? seed) |
Samples a point cloud from the shape. |
ValueTuple<Tensor<T>, Tensor<T>> ExtractMesh(Tensor<T> latent, int resolution) |
Extracts a mesh from the latent using marching cubes. |