BrushNetXModel<T>

Models & Types · AiDotNet.Diffusion.ImageEditing

BrushNet-X model extending BrushNet to SDXL architecture for high-resolution inpainting.

Inheritance: DiffusionModelBase<T> → LatentDiffusionModelBase<T> → BrushNetXModel<T>

For Beginners

This is a higher-resolution version of BrushNet built on SDXL. It generates larger, more detailed inpainting results at 1024x1024 resolution compared to BrushNet’s 512x512, with better text understanding from dual encoders.

How It Works

BrushNet-X adapts the dual-branch BrushNet inpainting approach to the SDXL architecture, enabling high-resolution (1024x1024) inpainting with improved coherence and detail. Uses SDXL’s dual text encoders (CLIP-L + CLIP-G) for enhanced prompt understanding.

Technical specifications:

  • Architecture: Dual-branch U-Net (SDXL backbone)
  • Text encoders: CLIP ViT-L/14 (768-dim) + OpenCLIP ViT-G/14 (1280-dim) = 2048 context
  • Input channels: 9 (4 latent + 4 masked image latent + 1 mask)
  • Base channels: 320, multipliers [1, 2, 4]
  • Resolution: 1024x1024 native

Example

var options = new DiffusionModelOptions<float> { LatentChannels = 4, DefaultInferenceSteps = 30 };
var input = Tensor<float>.CreateRandom(new[] { 1, 4, 128, 128 });
var trainX = Tensor<float>.CreateRandom(4, 8);
var trainY = Tensor<float>.CreateRandom(4, 2);
var result = new AiModelBuilder<float, Tensor<float>, Tensor<float>>()
    .ConfigureModel(new BrushNetXModel<float>(options: options))
    .Build(trainX, trainY);
var inpainted = result.Predict(input);

Constructors

Constructor Summary
BrushNetXModel(NeuralNetworkArchitecture<T> architecture, DiffusionModelOptions<T> options, INoiseScheduler<T> scheduler, UNetNoisePredictor<T> predictor, StandardVAE<T> vae, IConditioningModule<T> conditioner, int? seed)

Properties

Property Summary
IConditioningModule<T> Conditioner { get; }
INoisePredictor<T> NoisePredictor { get; }
IVAEModel<T> VAE { get; }
int LatentChannels { get; }

Methods

Method Summary
ModelMetadata<T> GetModelMetadata()