StableDiffusion2Model<T>
Models & Types · AiDotNet.Diffusion.TextToImage
Stable Diffusion 2.0/2.1 model for text-to-image generation.
Inheritance: DiffusionModelBase<T> → LatentDiffusionModelBase<T> → StableDiffusion2Model<T>
For Beginners
SD 2.0/2.1 is an upgraded version of SD 1.5 with key differences:
How SD 2.x works:
- Your text prompt is encoded by OpenCLIP ViT-H/14 (1024-dim embeddings)
- These embeddings guide a U-Net (865M parameters) that denoises in latent space
- A VAE decodes the denoised latent into a 768x768 image (SD 2.0) or 512/768 (SD 2.1)
Key differences from SD 1.5:
- Text encoder: OpenCLIP ViT-H/14 (1024-dim) vs CLIP ViT-L/14 (768-dim)
- Prediction type: v-prediction vs epsilon-prediction
- Native resolution: 768x768 (SD 2.0) or 512x512/768x768 (SD 2.1)
- Removed NSFW content from training data
- Better at generating text in images
When to use SD 2.x:
- Need v-prediction for certain workflows
- Want OpenCLIP text encoder (different strengths than CLIP)
- Need 768x768 native resolution
Limitations:
- Smaller community ecosystem than SD 1.5
- Some users prefer SD 1.5’s CLIP encoder for prompt adherence
- Not as widely supported by third-party tools
How It Works
Stable Diffusion 2.x is the second generation of Stability AI’s text-to-image model. It uses OpenCLIP ViT-H/14 instead of the original CLIP ViT-L/14 text encoder, and introduces v-prediction as the default prediction type.
Technical specifications:
- Architecture: U-Net with cross-attention and time embedding
- Text encoder: OpenCLIP ViT-H/14 (632M parameters, 1024-dim, 77 max tokens)
- U-Net: 865M parameters, base channels 320, [1, 2, 4, 4] multipliers
- Cross-attention dimension: 1024 (matches OpenCLIP output)
- VAE: KL-regularized autoencoder, 4 latent channels, scale factor 0.18215
- Noise schedule: Scaled linear beta schedule, 1000 training timesteps
- Prediction type: v-prediction (velocity prediction)
Reference: Rombach et al., “High-Resolution Image Synthesis with Latent Diffusion Models”, CVPR 2022
Example
// Create with industry-standard defaults
var sd2 = new StableDiffusion2Model<float>();
// Generate a 768x768 image from text
var image = sd2.GenerateFromText(
prompt: "A photograph of a mountain landscape at golden hour",
negativePrompt: "blurry, low quality, distorted",
width: 768,
height: 768,
numInferenceSteps: 30,
guidanceScale: 7.5,
seed: 42);
Constructors
| Constructor | Summary |
|---|---|
StableDiffusion2Model(NeuralNetworkArchitecture<T> architecture, DiffusionModelOptions<T> options, INoiseScheduler<T> scheduler, UNetNoisePredictor<T> unet, StandardVAE<T> vae, IConditioningModule<T> conditioner, bool useVPrediction, int? seed) |
Initializes a new instance of StableDiffusion2Model with full customization support. |
Properties
| Property | Summary |
|---|---|
IConditioningModule<T> Conditioner { get; } |
|
INoisePredictor<T> NoisePredictor { get; } |
|
IVAEModel<T> VAE { get; } |
|
bool UsesVPrediction { get; } |
Gets whether this model uses v-prediction (velocity prediction). |
int CrossAttentionDim { get; } |
Gets the cross-attention dimension (1024 for SD 2.x, matching OpenCLIP ViT-H/14). |
int LatentChannels { get; } |
Methods
| Method | Summary |
|---|---|
ModelMetadata<T> GetModelMetadata() |
|
Tensor<T> GenerateFromText(string prompt, string negativePrompt, int width, int height, int numInferenceSteps, double? guidanceScale, int? seed) |
|
Tensor<T> ImageToImage(Tensor<T> inputImage, string prompt, string negativePrompt, double strength, int numInferenceSteps, double? guidanceScale, int? seed) |
Fields
| Field | Summary |
|---|---|
int DefaultHeight |
Default image height for SD 2.x generation. |
int DefaultWidth |
Default image width for SD 2.x generation. |