AnyEditModel<T>

Models & Types · AiDotNet.Diffusion.ImageEditing

AnyEdit model for handling diverse editing types through unified instruction understanding.

Inheritance: DiffusionModelBase<T> → LatentDiffusionModelBase<T> → AnyEditModel<T>

For Beginners

AnyEdit is versatile — it can handle almost any type of image edit you describe in words. Change colors, add objects, modify styles, or adjust specific regions, all with the same model and natural language instructions.

How It Works

AnyEdit supports a wide range of editing operations including local/global edits, style changes, action edits, and visual concept manipulation through a unified model trained on diverse editing instruction-image pairs. Based on InstructPix2Pix with SD1.5.

Technical specifications:

  • Base model: Stable Diffusion 1.5 (InstructPix2Pix variant)
  • Text encoder: CLIP ViT-L/14 (768-dim)
  • Input channels: 8 (4 latent + 4 source image latent)
  • Edit types: local, global, style, action, visual concept
  • Training: 2.5M diverse instruction-image pairs

Reference: Yu et al., “AnyEdit: Mastering Unified High-Quality Image Editing for Any Idea”, 2024

Example

var options = new DiffusionModelOptions<float> { LatentChannels = 4, DefaultInferenceSteps = 30 };
var input = Tensor<float>.CreateRandom(new[] { 1, 4, 64, 64 });
var trainX = Tensor<float>.CreateRandom(4, 8);
var trainY = Tensor<float>.CreateRandom(4, 2);
var result = new AiModelBuilder<float, Tensor<float>, Tensor<float>>()
    .ConfigureModel(new AnyEditModel<float>(options: options))
    .Build(trainX, trainY);
var edited = result.Predict(input);

Constructors

Constructor Summary
AnyEditModel(NeuralNetworkArchitecture<T> architecture, DiffusionModelOptions<T> options, INoiseScheduler<T> scheduler, UNetNoisePredictor<T> predictor, StandardVAE<T> vae, IConditioningModule<T> conditioner, int? seed)

Properties

Property Summary
IConditioningModule<T> Conditioner { get; }
INoisePredictor<T> NoisePredictor { get; }
IVAEModel<T> VAE { get; }
int LatentChannels { get; }

Methods

Method Summary
ModelMetadata<T> GetModelMetadata()