SDEditModel<T>
Models & Types · AiDotNet.Diffusion.ImageEditing
SDEdit model for image-guided synthesis via partial noise injection and guided denoising.
Inheritance: DiffusionModelBase<T> → LatentDiffusionModelBase<T> → SDEditModel<T>
For Beginners
SDEdit turns rough sketches and images into polished results.
How it works:
- You provide an input image (sketch, rough edit, or photograph)
- The model adds noise to partially corrupt the image
- The diffusion model denoises it, guided by text prompts
- The result balances input structure with photorealistic output
Key characteristics:
- Based on Stable Diffusion 1.5 (512x512, CLIP ViT-L/14)
- No fine-tuning needed – works with any pre-trained diffusion model
- Noise strength controls realism vs. faithfulness tradeoff
- Uses DDPM scheduler for stochastic denoising
How It Works
SDEdit (Stochastic Differential Editing) transforms sketches, strokes, and existing images into realistic outputs by adding controlled amounts of noise to the input and then denoising with a diffusion model. The noise strength controls the balance between faithfulness to the input guide and realism of the output.
Technical specifications:
- Architecture: SD 1.5 U-Net with partial noise injection pipeline
- Text encoder: CLIP ViT-L/14 (768-dim, 77 max tokens)
- Cross-attention dimension: 768
- VAE: 4 latent channels, scale factor 0.18215
- Noise schedule: Scaled linear beta [0.00085, 0.012], 1000 timesteps
- Scheduler: DDPM (stochastic denoising for natural output variation)
Reference: Meng et al., “SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equations”, ICLR 2022
Example
var options = new DiffusionModelOptions<float> { LatentChannels = 4, DefaultInferenceSteps = 30 };
var input = Tensor<float>.CreateRandom(new[] { 1, 4, 64, 64 });
var trainX = Tensor<float>.CreateRandom(4, 8);
var trainY = Tensor<float>.CreateRandom(4, 2);
var result = new AiModelBuilder<float, Tensor<float>, Tensor<float>>()
.ConfigureModel(new SDEditModel<float>(options: options))
.Build(trainX, trainY);
var edited = result.Predict(input);
Constructors
| Constructor | Summary |
|---|---|
SDEditModel(NeuralNetworkArchitecture<T> architecture, DiffusionModelOptions<T> options, INoiseScheduler<T> scheduler, UNetNoisePredictor<T> unet, StandardVAE<T> vae, IConditioningModule<T> conditioner, int? seed) |
Initializes a new instance of SDEditModel with full customization support. |
Properties
| Property | Summary |
|---|---|
IConditioningModule<T> Conditioner { get; } |
|
INoisePredictor<T> NoisePredictor { get; } |
|
IVAEModel<T> VAE { get; } |
|
int CrossAttentionDim { get; } |
Gets the cross-attention dimension (768 for CLIP ViT-L/14). |
int LatentChannels { get; } |
Methods
| Method | Summary |
|---|---|
ModelMetadata<T> GetModelMetadata() |
|
Tensor<T> GenerateFromText(string prompt, string negativePrompt, int width, int height, int numInferenceSteps, double? guidanceScale, int? seed) |
|
Tensor<T> ImageToImage(Tensor<T> inputImage, string prompt, string negativePrompt, double strength, int numInferenceSteps, double? guidanceScale, int? seed) |
Fields
| Field | Summary |
|---|---|
int DefaultHeight |
Default image height for SDEdit (SD 1.5 native resolution). |
int DefaultWidth |
Default image width for SDEdit (SD 1.5 native resolution). |