StableDiffusion15Model<T>
Models & Types · AiDotNet.Diffusion.TextToImage
Stable Diffusion 1.5 model for text-to-image generation.
Inheritance: DiffusionModelBase<T> → LatentDiffusionModelBase<T> → StableDiffusion15Model<T>
For Beginners
Stable Diffusion 1.5 is the “classic” text-to-image AI model.
How SD 1.5 works:
- Your text prompt is encoded by a CLIP ViT-L/14 text encoder into embeddings
- These embeddings guide a U-Net (865M parameters) that denoises in latent space
- A VAE decodes the denoised latent into a 512x512 image
Key characteristics:
- Single text encoder: CLIP ViT-L/14 (768-dim embeddings)
- U-Net: 865M parameters, channel multipliers [1, 2, 4, 4]
- VAE: 4-channel latent space, 8x spatial downsampling
- Native resolution: 512x512 pixels
- Latent scale factor: 0.18215
- Guidance scale: 7.5 (default)
When to use SD 1.5:
- Huge community model ecosystem (thousands of fine-tunes available)
- Lower resource requirements than SDXL (runs on 4GB+ VRAM)
- Fast generation (20-50 steps)
- Excellent for 512x512 generation
Limitations:
- Lower resolution than SDXL (512x512 vs 1024x1024)
- Single text encoder (less prompt understanding than dual-encoder models)
- Occasional artifacts at hands, text, and complex scenes
How It Works
Stable Diffusion 1.5 (SD 1.5) is a latent diffusion model developed by Stability AI and Runway ML. It is the most widely used open-source text-to-image model and the foundation for an enormous ecosystem of fine-tunes, LoRAs, ControlNets, and community models.
Technical specifications:
- Architecture: U-Net with cross-attention and time embedding
- Text encoder: CLIP ViT-L/14 (63M parameters, 768-dim, 77 max tokens)
- U-Net: 865M parameters, base channels 320, [1, 2, 4, 4] multipliers
- Cross-attention dimension: 768 (matches CLIP output)
- Attention resolutions: at 4x, 2x, and 1x downsampling levels
- VAE: KL-regularized autoencoder, 4 latent channels, scale factor 0.18215
- Noise schedule: Scaled linear beta schedule, 1000 training timesteps
- Beta range: [0.00085, 0.012]
- Prediction type: Epsilon (noise prediction)
Reference: Rombach et al., “High-Resolution Image Synthesis with Latent Diffusion Models”, CVPR 2022
Example
var maskTensor = Tensor<float>.CreateRandom(1, 3, 32, 32);
var existingImage = Tensor<float>.CreateRandom(1, 3, 32, 32);
// Create with industry-standard defaults
var sd15 = new StableDiffusion15Model<float>();
// Swap in your own components. Every parameter is optional, so supply only what you are replacing;
// a conditioner is omitted here because CLIPTextConditioner needs a tokenizer, which needs a
// vocabulary — worth its own example rather than three lines of setup in this one.
var customSd15 = new StableDiffusion15Model<float>(
unet: new UNetNoisePredictor<float>(),
vae: new StandardVAE<float>());
// Generate a 512x512 image from text
var image = sd15.GenerateFromText(
prompt: "A photograph of a cat sitting on a windowsill at sunset",
negativePrompt: "blurry, low quality, distorted",
width: 512,
height: 512,
numInferenceSteps: 30,
guidanceScale: 7.5,
seed: 42);
// Image-to-image transformation
var transformed = sd15.ImageToImage(
inputImage: existingImage,
prompt: "Oil painting style, vibrant colors",
strength: 0.7,
numInferenceSteps: 30);
// Inpainting
var inpainted = sd15.Inpaint(
inputImage: existingImage,
mask: maskTensor,
prompt: "A fluffy golden retriever",
numInferenceSteps: 30);
Constructors
| Constructor | Summary |
|---|---|
StableDiffusion15Model(NeuralNetworkArchitecture<T> architecture, DiffusionModelOptions<T> options, INoiseScheduler<T> scheduler, UNetNoisePredictor<T> unet, StandardVAE<T> vae, IConditioningModule<T> conditioner, int? seed) |
Initializes a new instance of StableDiffusion15Model with full customization support. |
Properties
| Property | Summary |
|---|---|
IConditioningModule<T> Conditioner { get; } |
|
INoisePredictor<T> NoisePredictor { get; } |
|
IVAEModel<T> VAE { get; } |
|
int CrossAttentionDim { get; } |
Gets the cross-attention dimension (768 for SD 1.5, matching CLIP ViT-L/14). |
int LatentChannels { get; } |
Methods
| Method | Summary |
|---|---|
List<Tensor<T>> GenerateVariations(string prompt, string negativePrompt, int count, int width, int height, int numInferenceSteps, double? guidanceScale, int? baseSeed) |
Generates multiple image variations from the same prompt using different seeds. |
ModelMetadata<T> GetModelMetadata() |
|
Tensor<T> GenerateFromText(string prompt, string negativePrompt, int width, int height, int numInferenceSteps, double? guidanceScale, int? seed) |
Generates an image from a text prompt using SD 1.5 defaults. |
Tensor<T> ImageToImage(Tensor<T> inputImage, string prompt, string negativePrompt, double strength, int numInferenceSteps, double? guidanceScale, int? seed) |
Performs image-to-image transformation using SD 1.5. |
Fields
| Field | Summary |
|---|---|
int DefaultHeight |
Default image height for SD 1.5 generation. |
int DefaultWidth |
Default image width for SD 1.5 generation. |