SDXLInpaintingModel<T>

Models & Types · AiDotNet.Diffusion.ImageEditing

SDXL Inpainting model for high-resolution 1024x1024 mask-based image inpainting.

Inheritance: DiffusionModelBase<T> → LatentDiffusionModelBase<T> → SDXLInpaintingModel<T>

For Beginners

This is Stable Diffusion XL’s built-in inpainting model. You provide an image and paint a mask over what you want to change, then describe what should appear there. It generates results at 1024x1024 resolution with excellent text understanding from dual CLIP encoders.

How It Works

The official SDXL inpainting model uses a 9-channel input (4 latent + 4 masked image + 1 mask) with the full SDXL U-Net architecture. It provides native 1024x1024 inpainting with the quality and prompt following of SDXL’s dual text encoder conditioning.

Technical specifications:

  • Architecture: SDXL U-Net with 9-channel input
  • Text encoders: CLIP ViT-L/14 (768-dim) + OpenCLIP ViT-G/14 (1280-dim) = 2048 context
  • Input channels: 9 (4 latent + 4 masked image latent + 1 binary mask)
  • Base channels: 320, multipliers [1, 2, 4]
  • Resolution: 1024x1024 native
  • Scheduler: Euler Discrete (SDXL default)

Reference: Podell et al., “SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis”, ICLR 2024

Example

var options = new DiffusionModelOptions<float> { LatentChannels = 4, DefaultInferenceSteps = 30 };
var input = Tensor<float>.CreateRandom(new[] { 1, 4, 128, 128 });
var trainX = Tensor<float>.CreateRandom(4, 8);
var trainY = Tensor<float>.CreateRandom(4, 2);
var result = new AiModelBuilder<float, Tensor<float>, Tensor<float>>()
    .ConfigureModel(new SDXLInpaintingModel<float>(options: options))
    .Build(trainX, trainY);
var inpainted = result.Predict(input);

Constructors

Constructor Summary
SDXLInpaintingModel(NeuralNetworkArchitecture<T> architecture, DiffusionModelOptions<T> options, INoiseScheduler<T> scheduler, UNetNoisePredictor<T> predictor, StandardVAE<T> vae, IConditioningModule<T> conditioner, int? seed)

Properties

Property Summary
IConditioningModule<T> Conditioner { get; }
INoisePredictor<T> NoisePredictor { get; }
IVAEModel<T> VAE { get; }
int LatentChannels { get; }

Methods

Method Summary
ModelMetadata<T> GetModelMetadata()