BrushEditModel<T>
Models & Types · AiDotNet.Diffusion.ImageEditing
BrushEdit model combining multimodal LLM understanding with BrushNet inpainting.
Inheritance: DiffusionModelBase<T> → LatentDiffusionModelBase<T> → BrushEditModel<T>
For Beginners
BrushEdit combines AI understanding with inpainting. Instead of drawing masks yourself, you just describe what to change (“remove the car on the left”), and BrushEdit figures out where to mask and what to generate.
How It Works
BrushEdit uses a multimodal LLM to understand editing instructions and automatically generate appropriate masks, then applies BrushNet-style dual-branch inpainting on SD1.5. This enables natural language editing without manual mask creation.
Technical specifications:
- Pipeline: Multimodal LLM (mask generation) + BrushNet (dual-branch inpainting)
- Inpainting backbone: SD1.5 with BrushNet dual-branch architecture
- Text encoder: CLIP ViT-L/14 (768-dim)
- Input channels: 9 (4 latent + 4 masked image latent + 1 mask)
- Mask generation: Automatic from natural language via LLM
Reference: Li et al., “BrushEdit: All-In-One Image Inpainting and Editing”, 2024
Example
var options = new DiffusionModelOptions<float> { LatentChannels = 4, DefaultInferenceSteps = 30 };
var input = Tensor<float>.CreateRandom(new[] { 1, 4, 64, 64 });
var trainX = Tensor<float>.CreateRandom(4, 8);
var trainY = Tensor<float>.CreateRandom(4, 2);
var result = new AiModelBuilder<float, Tensor<float>, Tensor<float>>()
.ConfigureModel(new BrushEditModel<float>(options: options))
.Build(trainX, trainY);
var edited = result.Predict(input);
Constructors
| Constructor | Summary |
|---|---|
BrushEditModel(NeuralNetworkArchitecture<T> architecture, DiffusionModelOptions<T> options, INoiseScheduler<T> scheduler, UNetNoisePredictor<T> predictor, StandardVAE<T> vae, IConditioningModule<T> conditioner, int? seed) |
Properties
| Property | Summary |
|---|---|
IConditioningModule<T> Conditioner { get; } |
|
INoisePredictor<T> NoisePredictor { get; } |
|
IVAEModel<T> VAE { get; } |
|
int LatentChannels { get; } |
Methods
| Method | Summary |
|---|---|
ModelMetadata<T> GetModelMetadata() |