StableDiffusion35Model<T>
Models & Types · AiDotNet.Diffusion.TextToImage
Stable Diffusion 3.5 model with improved MMDiT-X architecture by Stability AI.
Inheritance: DiffusionModelBase<T> → LatentDiffusionModelBase<T> → StableDiffusion35Model<T>
For Beginners
Stable Diffusion 3.5 is Stability AI’s latest open model.
How SD 3.5 works:
- Text is encoded by three encoders: CLIP ViT-L/14, OpenCLIP ViT-bigG, and T5-XXL
- An improved MMDiT-X transformer with QK-normalization processes text and image tokens
- Rectified flow matching enables efficient 28-40 step generation
- A 16-channel VAE decodes latents to high-resolution images
Model variants:
- SD 3.5 Medium: 2.5B parameters, faster, good quality-speed tradeoff
- SD 3.5 Large: 8B parameters, highest quality, more VRAM required
Key characteristics:
- MMDiT-X architecture with QK-normalization for stable training
- Triple text encoders for superior prompt understanding
- 16 latent channels with improved VAE
- Rectified flow matching (not DDPM/DDIM)
- Medium: 2.5B params, 1536 hidden, 24 layers
- Large: 8B params, 4096 hidden, 38 layers
Advantages:
- Open-weight (Stability AI Community License)
- State-of-the-art quality among open models
- Better prompt adherence than SDXL
- QK-norm prevents training instabilities
Limitations:
- Large variant requires significant VRAM (~24GB)
- Newer ecosystem than SD 1.5/SDXL
- Commercial use requires separate license agreement
How It Works
SD 3.5 builds on SD3’s MMDiT architecture with improved text-image alignment, better detail generation, and QK-normalization for training stability. Uses rectified flow matching training and triple text encoders for comprehensive prompt understanding. Available in Medium (2.5B) and Large (8B) variants.
Technical specifications:
- Architecture: MMDiT-X with QK-normalization
- Medium: 2.5B params, hidden 1536, 24 layers, 24 heads
- Large: 8B params, hidden 4096, 38 layers, 64 heads
- Text encoder 1: CLIP ViT-L/14 (768-dim, pooled)
- Text encoder 2: OpenCLIP ViT-bigG (1280-dim, pooled)
- Text encoder 3: T5-XXL (4096-dim, sequence)
- Patch size: 2 (in latent space)
- VAE: 16 latent channels, 8x spatial compression
- Training: Rectified flow matching
- Resolution: 1024x1024 default, up to 2048x2048
Reference: Esser et al., “Scaling Rectified Flow Transformers for High-Resolution Image Synthesis”, ICML 2024 (SD3 base); Stability AI, “Stable Diffusion 3.5 Release Notes”, 2024
Example
// Create SD 3.5 Medium (default)
var sd35 = new StableDiffusion35Model<float>();
// Create SD 3.5 Large for higher quality
var sd35Large = new StableDiffusion35Model<float>(variant: MMDiTXVariant.Large);
// Generate an image
var image = sd35.GenerateFromText(
prompt: "A detailed oil painting of a medieval castle at dawn",
width: 1024,
height: 1024,
numInferenceSteps: 28,
guidanceScale: 7.0,
seed: 42);
Constructors
| Constructor | Summary |
|---|---|
StableDiffusion35Model(NeuralNetworkArchitecture<T> architecture, DiffusionModelOptions<T> options, INoiseScheduler<T> scheduler, MMDiTXNoisePredictor<T> predictor, StandardVAE<T> vae, IConditioningModule<T> conditioner, MMDiTXVariant variant, int? seed) |
Initializes a new instance of StableDiffusion35Model. |
Properties
| Property | Summary |
|---|---|
IConditioningModule<T> Conditioner { get; } |
|
INoisePredictor<T> NoisePredictor { get; } |
|
IVAEModel<T> VAE { get; } |
|
MMDiTXVariant Variant { get; } |
Gets the model variant (Medium or Large). |
int LatentChannels { get; } |
Methods
| Method | Summary |
|---|---|
ModelMetadata<T> GetModelMetadata() |
|
Tensor<T> GenerateFromText(string prompt, string negativePrompt, int width, int height, int numInferenceSteps, double? guidanceScale, int? seed) |
|
Tensor<T> ImageToImage(Tensor<T> inputImage, string prompt, string negativePrompt, double strength, int numInferenceSteps, double? guidanceScale, int? seed) |
Fields
| Field | Summary |
|---|---|
int DefaultHeight |
Default image height for SD 3.5 (1024x1024). |
int DefaultWidth |
Default image width for SD 3.5 (1024x1024). |