LGMModel<T>

Models & Types · AiDotNet.Diffusion.ThreeD

LGM (Large Gaussian Model) for feed-forward 3D Gaussian generation from multi-view images.

Inheritance: DiffusionModelBase<T> → LatentDiffusionModelBase<T> → ThreeDDiffusionModelBase<T> → LGMModel<T>

For Beginners

LGM creates 3D Gaussian splats instantly from multi-view images.

How LGM works:

  1. Generate 4 views of the object using a multi-view diffusion model
  2. Encode views with DINO-v2 into 1024-dim features
  3. Concatenate 4 view latents (4x4 = 16 input channels)
  4. Asymmetric U-Net predicts 14 Gaussian parameters per pixel
  5. Unproject pixel predictions to 3D Gaussian positions
  6. Result: 50,000+ 3D Gaussians in ~5 seconds

Key characteristics:

  • Feed-forward: single pass, no optimization loop
  • ~5 seconds on GPU (vs minutes for SDS-based methods)
  • 50,000 Gaussians for high-quality reconstruction
  • Asymmetric U-Net: large decoder for detail, compact encoder
  • Real-time Gaussian splatting rendering

When to use LGM:

  • Real-time 3D content generation
  • Interactive 3D from images
  • High-throughput 3D asset creation
  • When speed is more important than maximum quality

Limitations:

  • Requires multi-view input (needs separate multi-view generation)
  • Quality depends on multi-view consistency
  • Fixed number of Gaussians per prediction
  • Less detail than optimization-based methods

How It Works

LGM uses a large asymmetric U-Net backbone to predict 3D Gaussians from multi-view images in a single forward pass, enabling real-time 3D generation without per-shape optimization.

Architecture components:

  • Asymmetric U-Net for Gaussian parameter prediction (64 base channels, [1,2,4,8])
  • 14 output channels per pixel (position 3 + color 3 + opacity 1 + covariance 7)
  • 16 input channels (4 latent channels x 4 views)
  • DINO-v2 image encoder for 1024-dim conditioning
  • Standard SD VAE for multi-view image encoding
  • Feed-forward: no iterative optimization required

Technical specifications:

  • Architecture: Asymmetric U-Net for Gaussian prediction
  • Input channels: 16 (4 views x 4 latent channels)
  • Output channels: 14 (position + color + opacity + covariance)
  • Base channels: 64, multipliers [1, 2, 4, 8]
  • Attention resolutions: [8, 4]
  • Image encoder: DINO-v2 (1024-dim)
  • Default Gaussians: 50,000
  • Generation time: ~5 seconds
  • Feed-forward: Yes (no iterative optimization)

Reference: Tang et al., “LGM: Large Multi-View Gaussian Model for High-Resolution 3D Content Creation”, ECCV 2024

Example

// Create with industry-standard defaults
var lgm = new LGMModel<float>();
            
// Generate 3D Gaussian point cloud from multi-view images
var points = lgm.GeneratePointCloud(
    prompt: "A detailed chess piece",
    numPoints: 50000,
    numInferenceSteps: 1,
    guidanceScale: 1.0);
            
// Generate textured mesh
var mesh = lgm.GenerateMesh(
    prompt: "A toy car",
    resolution: 256,
    numInferenceSteps: 1);

Constructors

Constructor Summary
LGMModel(NeuralNetworkArchitecture<T> architecture, DiffusionModelOptions<T> options, INoiseScheduler<T> scheduler, UNetNoisePredictor<T> unet, StandardVAE<T> vae, IConditioningModule<T> conditioner, int defaultPointCount, int? seed) Initializes a new instance of LGMModel with full customization support.

Properties

Property Summary
IConditioningModule<T> Conditioner { get; }
INoisePredictor<T> NoisePredictor { get; }
IVAEModel<T> VAE { get; }
bool SupportsMesh { get; }
bool SupportsNovelView { get; }
bool SupportsPointCloud { get; }
bool SupportsScoreDistillation { get; }
bool SupportsTexture { get; }
int LatentChannels { get; }

Methods

Method Summary
Mesh3D<T> GenerateMesh(string prompt, string negativePrompt, int resolution, int numInferenceSteps, double guidanceScale, int? seed)
ModelMetadata<T> GetModelMetadata()
Tensor<T> GeneratePointCloud(string prompt, string negativePrompt, int? numPoints, int numInferenceSteps, double guidanceScale, int? seed)