ReplaceAnythingModel<T>
Models & Types · AiDotNet.Diffusion.ImageEditing
ReplaceAnything model for text-guided object replacement within SAM-segmented regions.
Inheritance: DiffusionModelBase<T> → LatentDiffusionModelBase<T> → ReplaceAnythingModel<T>
For Beginners
Click on an object in your image (a car, a tree, a person), describe what you want instead (“a red sports car”, “a cherry blossom tree”), and ReplaceAnything seamlessly swaps it out while keeping the rest of the image intact.
How It Works
Combines SAM-based segmentation with SD1.5 inpainting diffusion to automatically replace any segmented object with text-described content. The user selects an object (via point, box, or mask) and provides a text description of the replacement.
Technical specifications:
- Pipeline: SAM segmentation -> mask generation -> SD1.5 inpainting
- Segmentation: Segment Anything Model (SAM) for interactive selection
- Inpainting: SD1.5 with 9-channel input (4 latent + 4 masked + 1 mask)
- Text encoder: CLIP ViT-L/14 (768-dim)
- Selection modes: point, bounding box, or manual mask
Example
var options = new DiffusionModelOptions<float> { LatentChannels = 4, DefaultInferenceSteps = 30 };
var input = Tensor<float>.CreateRandom(new[] { 1, 4, 64, 64 });
var trainX = Tensor<float>.CreateRandom(4, 8);
var trainY = Tensor<float>.CreateRandom(4, 2);
var result = new AiModelBuilder<float, Tensor<float>, Tensor<float>>()
.ConfigureModel(new ReplaceAnythingModel<float>(options: options))
.Build(trainX, trainY);
var prediction = result.Predict(input);
Constructors
| Constructor | Summary |
|---|---|
ReplaceAnythingModel(NeuralNetworkArchitecture<T> architecture, DiffusionModelOptions<T> options, INoiseScheduler<T> scheduler, UNetNoisePredictor<T> predictor, StandardVAE<T> vae, IConditioningModule<T> conditioner, int? seed) |
Properties
| Property | Summary |
|---|---|
IConditioningModule<T> Conditioner { get; } |
|
INoisePredictor<T> NoisePredictor { get; } |
|
IVAEModel<T> VAE { get; } |
|
int LatentChannels { get; } |
Methods
| Method | Summary |
|---|---|
ModelMetadata<T> GetModelMetadata() |