CycleGAN<T>

Models & Types · AiDotNet.NeuralNetworks

Represents a CycleGAN for unpaired image-to-image translation.

Inheritance: NeuralNetworkBase<T> → DeclaredModelLayoutBase<T> → ImageTranslationModelLayoutBase<T> → CycleGAN<T>

For Beginners

CycleGAN translates images without matched pairs.

Key innovation:

  • Doesn’t need paired training data
  • Learns from two separate collections of images
  • Example: Photos of horses + Photos of zebras → can convert horses to zebras

How it works:

  • Two generators: G (A→B) and F (B→A)
  • Two discriminators: D_A and D_B
  • Cycle consistency: G(F(B)) ≈ B and F(G(A)) ≈ A
  • This prevents mode collapse and maintains content

Applications:

  • Style transfer (Monet → Photo, Photo → Monet)
  • Season transfer (Summer → Winter)
  • Object transfiguration (Horse → Zebra)
  • Domain adaptation

Reference: Zhu et al., “Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks” (2017)

How It Works

CycleGAN enables image-to-image translation without paired training data:

  • Uses two generators (A→B and B→A) and two discriminators
  • Enforces cycle consistency: A→B→A should equal A
  • Works without paired examples (e.g., can learn horses→zebras from separate collections)
  • Uses adversarial loss + cycle consistency loss + identity loss

Example

var input = Tensor<float>.CreateRandom(new[] { 1, 3, 256, 256 });
var trainX = Tensor<float>.CreateRandom(4, 8);
var trainY = Tensor<float>.CreateRandom(4, 2);
var result = new AiModelBuilder<float, Tensor<float>, Tensor<float>>()
    .ConfigureModel(new CycleGAN<float>())
    .Build(trainX, trainY);
var translated = result.Predict(input);

Constructors

Constructor Summary
CycleGAN() Initializes a new instance of the CycleGAN class with the specified architecture and training parameters.
CycleGAN(NeuralNetworkArchitecture<T> generatorAtoB, NeuralNetworkArchitecture<T> generatorBtoA, NeuralNetworkArchitecture<T> discriminatorA, NeuralNetworkArchitecture<T> discriminatorB, InputType inputType, CycleGANOptions options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> generatorAtoBOptimizer, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> generatorBtoAOptimizer, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> discriminatorAOptimizer, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> discriminatorBOptimizer, ILossFunction<T> lossFunction)

Properties

Property Summary
NeuralNetworkBase<T> DiscriminatorA { get; set; } Discriminator for domain A.
NeuralNetworkBase<T> DiscriminatorB { get; set; } Discriminator for domain B.
NeuralNetworkBase<T> GeneratorAtoB { get; set; } Generator A→B.
NeuralNetworkBase<T> GeneratorBtoA { get; set; } Generator B→A.

Methods

Method Summary
Dictionary<string, Tensor<T>> GetNamedLayerActivations(Tensor<T> input)
ModelMetadata<T> GetModelMetadata()
Tensor<T> ForwardForTraining(Tensor<T> input) Defines the CycleGAN forward graph for tape-based training.
Tensor<T> TranslateAtoB(Tensor<T> imageA) Translates image from domain A to domain B.
Tensor<T> TranslateBtoA(Tensor<T> imageB) Translates image from domain B to domain A.
ValueTuple<T, T, T> TrainStep(Tensor<T> realA, Tensor<T> realB) Performs one training step for CycleGAN.
void ResetOptimizerState() Resets the state of all optimizers to their initial values.
void SetTrainingMode(bool isTraining) Forwards the mode to the four sub-networks, which the base walk cannot reach.