Pix2PixZeroModel<T>
Models & Types · AiDotNet.Diffusion.ImageEditing
Pix2Pix-Zero model for zero-shot image-to-image translation without paired training data.
Inheritance: DiffusionModelBase<T> → LatentDiffusionModelBase<T> → Pix2PixZeroModel<T>
For Beginners
Pix2Pix-Zero can translate images between domains (like “cat to dog” or “day to night”) without needing matched training pairs. It figures out the transformation direction automatically from the text descriptions alone.
How It Works
Pix2Pix-Zero performs image translation by discovering editing directions in the text embedding space using sentence similarity, then applying these directions during DDIM inversion-based editing. No paired training examples needed. Uses standard SD1.5 backbone with cross-attention guidance.
Technical specifications:
- Base model: Stable Diffusion 1.5
- Text encoder: CLIP ViT-L/14 (768-dim)
- Input channels: 4 (standard latent)
- Editing: DDIM inversion + text embedding direction discovery
- Cross-attention guidance for structural preservation
Reference: Parmar et al., “Zero-shot Image-to-Image Translation”, SIGGRAPH 2023
Example
var options = new DiffusionModelOptions<float> { LatentChannels = 4, DefaultInferenceSteps = 30 };
var input = Tensor<float>.CreateRandom(new[] { 1, 4, 64, 64 });
var trainX = Tensor<float>.CreateRandom(4, 8);
var trainY = Tensor<float>.CreateRandom(4, 2);
var result = new AiModelBuilder<float, Tensor<float>, Tensor<float>>()
.ConfigureModel(new Pix2PixZeroModel<float>(options: options))
.Build(trainX, trainY);
var translated = result.Predict(input);
Constructors
| Constructor | Summary |
|---|---|
Pix2PixZeroModel(NeuralNetworkArchitecture<T> architecture, DiffusionModelOptions<T> options, INoiseScheduler<T> scheduler, UNetNoisePredictor<T> predictor, StandardVAE<T> vae, IConditioningModule<T> conditioner, int? seed) |
Properties
| Property | Summary |
|---|---|
IConditioningModule<T> Conditioner { get; } |
|
INoisePredictor<T> NoisePredictor { get; } |
|
IVAEModel<T> VAE { get; } |
|
int LatentChannels { get; } |
Methods
| Method | Summary |
|---|---|
ModelMetadata<T> GetModelMetadata() |