Pix2Pix<T>
Models & Types · AiDotNet.NeuralNetworks
Represents a Pix2Pix GAN for paired image-to-image translation tasks.
Inheritance: NeuralNetworkBase<T> → DeclaredModelLayoutBase<T> → ImageTranslationModelLayoutBase<T> → Pix2Pix<T>
For Beginners
Pix2Pix transforms one type of image to another.
Key features:
- Learns from paired examples (input A becomes output B)
- Generator: U-Net architecture preserves spatial information
- Discriminator: PatchGAN focuses on local image patches
- Loss: Both “looks real” and “matches input”
Example use cases:
- Convert sketches to realistic photos
- Colorize black-and-white images
- Transform day scenes to night
- Semantic labels to photorealistic images
- Map to satellite image
Reference: Isola et al., “Image-to-Image Translation with Conditional Adversarial Networks” (2017)
How It Works
Pix2Pix is a conditional GAN for paired image-to-image translation:
- Uses a U-Net generator with skip connections
- Uses a PatchGAN discriminator that classifies image patches
- Combines adversarial loss with L1 reconstruction loss
- Requires paired training data (input-output pairs)
- Works for various tasks: edges to photo, day to night, sketch to image, etc.
Example
var options = new Pix2PixOptions { };
var input = Tensor<float>.CreateRandom(new[] { 1, 3, 256, 256 });
var trainX = Tensor<float>.CreateRandom(4, 8);
var trainY = Tensor<float>.CreateRandom(4, 2);
var generatorArchitecture = new NeuralNetworkArchitecture<float>(inputFeatures: 8, outputSize: 8);
var discriminatorArchitecture = new NeuralNetworkArchitecture<float>(inputFeatures: 8, outputSize: 1);
var result = new AiModelBuilder<float, Tensor<float>, Tensor<float>>()
.ConfigureModel(new Pix2Pix<float>(
generatorArchitecture, discriminatorArchitecture,
inputType: InputType.ThreeDimensional))
.Build(trainX, trainY);
var translated = result.Predict(input);
Constructors
| Constructor | Summary |
|---|---|
Pix2Pix(NeuralNetworkArchitecture<T> generatorArchitecture, NeuralNetworkArchitecture<T> discriminatorArchitecture, InputType inputType, Pix2PixOptions options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> generatorOptimizer, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> discriminatorOptimizer, ILossFunction<T> lossFunction) |
Initializes a new instance of the Pix2Pix class. |
Properties
| Property | Summary |
|---|---|
NeuralNetworkBase<T> Discriminator { get; set; } |
Gets the PatchGAN discriminator network. |
NeuralNetworkBase<T> Generator { get; set; } |
Gets the U-Net generator network. |
Methods
| Method | Summary |
|---|---|
Dictionary<string, Tensor<T>> GetNamedLayerActivations(Tensor<T> input) |
|
ModelMetadata<T> GetModelMetadata() |
|
Tensor<T> Translate(Tensor<T> inputImages) |
Translates input images to output images. |
ValueTuple<T, T, T> TrainStep(Tensor<T> inputImages, Tensor<T> targetImages) |
Performs one training step for Pix2Pix. |
void ResetOptimizerState() |
Resets both optimizer states for a fresh training run. |