SemanticScoreDistillation<T>

Models & Types · AiDotNet.Diffusion.Distillation

Semantic Score Distillation (SemanticSDS) for semantically-guided 3D generation.

For Beginners

Sometimes SDS produces 3D objects that look nice but don’t match the text description well (semantic drift). SemanticSDS splits the optimization into “does it mean the right thing?” and “does it look good?”, optimizing both separately. This ensures the 3D model of a “red ceramic mug” actually looks like a mug, not just a red blob.

How It Works

SemanticSDS enhances score distillation by incorporating semantic understanding from vision-language models (e.g., CLIP). It decomposes the SDS gradient into semantic and appearance components, allowing independent control over what the 3D object represents versus how it looks. This reduces semantic drift and produces 3D objects that better match the text prompt’s meaning.

Reference: Adapted from semantic-aware score distillation techniques combining CLIP guidance with diffusion model scores for text-to-3D generation

Constructors

Constructor Summary
SemanticScoreDistillation(IDiffusionModel<T> diffusionModel, double guidanceScale, double semanticWeight, double appearanceWeight) Initializes a new SemanticSDS instance.

Properties

Property Summary
double AppearanceWeight { get; } Gets the weight for the appearance component.
double GuidanceScale { get; } Gets the guidance scale.
double SemanticWeight { get; } Gets the weight for the semantic component.

Methods

Method Summary
T ComputeSemanticAlignment(Vector<T> renderEmbedding, Vector<T> textEmbedding) Computes the semantic alignment score between render embedding and text embedding.
Vector<T> ComputeGradient(Vector<T> sdsGradient, Vector<T> semanticGradient) Computes the semantic SDS gradient with decomposed components.