LEDITSPPModel<T>

Models & Types · AiDotNet.Diffusion.ImageEditing

LEDITS++ model for precise multi-concept editing of real images with automatic masking.

Inheritance: DiffusionModelBase<T> → LatentDiffusionModelBase<T> → LEDITSPPModel<T>

For Beginners

LEDITS++ lets you edit multiple things in a photo at once.

How it works:

  1. The real image is inverted into latent space using DDPM-based inversion
  2. Multiple editing prompts can be applied simultaneously
  3. Automatic masking determines which regions each edit affects
  4. Edits are blended using semantic guidance for natural results

Key characteristics:

  • Based on Stable Diffusion 1.5 (512x512, CLIP ViT-L/14)
  • Supports multiple simultaneous edits (e.g., change hair AND add glasses)
  • Automatic semantic masking per concept
  • Uses DPM-Solver++ for fast, high-quality sampling (15-25 steps)

How It Works

LEDITS++ (Lightweight and Efficient Diffusion Inter-Image Transformations with Semantic guidance++) enables simultaneous editing of multiple concepts in real images. It combines DDPM inversion with semantic grounding to automatically determine which image regions correspond to each editing concept, enabling precise multi-edit operations without manual masks.

Technical specifications:

  • Architecture: SD 1.5 U-Net with multi-concept semantic guidance
  • Text encoder: CLIP ViT-L/14 (768-dim, 77 max tokens)
  • Cross-attention dimension: 768
  • VAE: 4 latent channels, scale factor 0.18215
  • Noise schedule: Scaled linear beta [0.00085, 0.012], 1000 timesteps
  • Scheduler: DPM-Solver++ multistep (fast convergence)

Reference: Brack et al., “LEDITS++: Limitless Image Editing using Text-to-Image Models”, CVPR 2024

Example

var options = new DiffusionModelOptions<float> { LatentChannels = 4, DefaultInferenceSteps = 30 };
var input = Tensor<float>.CreateRandom(new[] { 1, 4, 64, 64 });
var trainX = Tensor<float>.CreateRandom(4, 8);
var trainY = Tensor<float>.CreateRandom(4, 2);
var result = new AiModelBuilder<float, Tensor<float>, Tensor<float>>()
    .ConfigureModel(new LEDITSPPModel<float>(options: options))
    .Build(trainX, trainY);
var edited = result.Predict(input);

Constructors

Constructor Summary
LEDITSPPModel(NeuralNetworkArchitecture<T> architecture, DiffusionModelOptions<T> options, INoiseScheduler<T> scheduler, UNetNoisePredictor<T> unet, StandardVAE<T> vae, IConditioningModule<T> conditioner, int? seed) Initializes a new instance of LEDITSPPModel with full customization support.

Properties

Property Summary
IConditioningModule<T> Conditioner { get; }
INoisePredictor<T> NoisePredictor { get; }
IVAEModel<T> VAE { get; }
int CrossAttentionDim { get; } Gets the cross-attention dimension (768 for CLIP ViT-L/14).
int LatentChannels { get; }

Methods

Method Summary
ModelMetadata<T> GetModelMetadata()
Tensor<T> GenerateFromText(string prompt, string negativePrompt, int width, int height, int numInferenceSteps, double? guidanceScale, int? seed)
Tensor<T> ImageToImage(Tensor<T> inputImage, string prompt, string negativePrompt, double strength, int numInferenceSteps, double? guidanceScale, int? seed)

Fields

Field Summary
int DefaultHeight Default image height for LEDITS++ (SD 1.5 native resolution).
int DefaultWidth Default image width for LEDITS++ (SD 1.5 native resolution).