SDXLInpaintingModel<T>
Models & Types · AiDotNet.Diffusion.ImageEditing
SDXL Inpainting model for high-resolution 1024x1024 mask-based image inpainting.
Inheritance: DiffusionModelBase<T> → LatentDiffusionModelBase<T> → SDXLInpaintingModel<T>
For Beginners
This is Stable Diffusion XL’s built-in inpainting model. You provide an image and paint a mask over what you want to change, then describe what should appear there. It generates results at 1024x1024 resolution with excellent text understanding from dual CLIP encoders.
How It Works
The official SDXL inpainting model uses a 9-channel input (4 latent + 4 masked image + 1 mask) with the full SDXL U-Net architecture. It provides native 1024x1024 inpainting with the quality and prompt following of SDXL’s dual text encoder conditioning.
Technical specifications:
- Architecture: SDXL U-Net with 9-channel input
- Text encoders: CLIP ViT-L/14 (768-dim) + OpenCLIP ViT-G/14 (1280-dim) = 2048 context
- Input channels: 9 (4 latent + 4 masked image latent + 1 binary mask)
- Base channels: 320, multipliers [1, 2, 4]
- Resolution: 1024x1024 native
- Scheduler: Euler Discrete (SDXL default)
Reference: Podell et al., “SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis”, ICLR 2024
Example
var options = new DiffusionModelOptions<float> { LatentChannels = 4, DefaultInferenceSteps = 30 };
var input = Tensor<float>.CreateRandom(new[] { 1, 4, 128, 128 });
var trainX = Tensor<float>.CreateRandom(4, 8);
var trainY = Tensor<float>.CreateRandom(4, 2);
var result = new AiModelBuilder<float, Tensor<float>, Tensor<float>>()
.ConfigureModel(new SDXLInpaintingModel<float>(options: options))
.Build(trainX, trainY);
var inpainted = result.Predict(input);
Constructors
| Constructor | Summary |
|---|---|
SDXLInpaintingModel(NeuralNetworkArchitecture<T> architecture, DiffusionModelOptions<T> options, INoiseScheduler<T> scheduler, UNetNoisePredictor<T> predictor, StandardVAE<T> vae, IConditioningModule<T> conditioner, int? seed) |
Properties
| Property | Summary |
|---|---|
IConditioningModule<T> Conditioner { get; } |
|
INoisePredictor<T> NoisePredictor { get; } |
|
IVAEModel<T> VAE { get; } |
|
int LatentChannels { get; } |
Methods
| Method | Summary |
|---|---|
ModelMetadata<T> GetModelMetadata() |