StableVITONModel<T>

Models & Types · AiDotNet.Diffusion.VirtualTryOn

StableVITON model for learning semantic correspondence with Stable Diffusion for virtual try-on.

Inheritance: DiffusionModelBase<T> → LatentDiffusionModelBase<T> → StableVITONModel<T>

For Beginners

StableVITON uses Stable Diffusion (a powerful image generator) to handle virtual try-on. It learns how garment parts correspond to body parts, producing very realistic results because it builds on SD’s strong image generation.

How It Works

StableVITON leverages pretrained Stable Diffusion with zero cross-attention blocks to learn spatial correspondence between garment and person images. Fine-tunes with ControlNet-like conditioning for realistic garment transfer.

Reference: Kim et al., “StableVITON: Learning Semantic Correspondence with Latent Diffusion Model for Virtual Try-On”, CVPR 2024

Example

var options = new DiffusionModelOptions<float> { LatentChannels = 4, DefaultInferenceSteps = 30 };
var input = Tensor<float>.CreateRandom(new[] { 1, 4, 128, 96 });
var trainX = Tensor<float>.CreateRandom(4, 8);
var trainY = Tensor<float>.CreateRandom(4, 2);
var result = new AiModelBuilder<float, Tensor<float>, Tensor<float>>()
    .ConfigureModel(new StableVITONModel<float>(options: options))
    .Build(trainX, trainY);
var tryOn = result.Predict(input);

Constructors

Constructor Summary
StableVITONModel(NeuralNetworkArchitecture<T> architecture, DiffusionModelOptions<T> options, INoiseScheduler<T> scheduler, UNetNoisePredictor<T> predictor, StandardVAE<T> vae, IConditioningModule<T> conditioner, int? seed)

Properties

Property Summary
IConditioningModule<T> Conditioner { get; }
INoisePredictor<T> NoisePredictor { get; }
IVAEModel<T> VAE { get; }
int LatentChannels { get; }

Methods

Method Summary
ModelMetadata<T> GetModelMetadata()