AdversarialLoss<T>

Models & Types · AiDotNet.LossFunctions

Adversarial (GAN) loss driven by a real discriminator, computed on the autodiff tape.

Inheritance: LossFunctionBase<T> → AdversarialLoss<T>

For Beginners

a GAN trains two networks against each other. The generator produces an image; the discriminator tries to tell generated images from real ones. This loss provides both halves of that game: ComputeTapeLoss is what the GENERATOR minimizes (it wants to be judged real), and ComputeDiscriminatorTapeLoss is what the DISCRIMINATOR minimizes (it wants to judge correctly). They are trained in alternation.

How It Works

Why this class exists. A GAN term is only real if the discriminator’s forward pass is part of the differentiated graph — the generator learns exclusively through the discriminator’s gradient. A “GAN loss” that scores a discriminator output with plain arithmetic outside the tape produces a plausible-looking number while sending the generator no signal at all, which is indistinguishable from having no adversarial term. Every operation here goes through Engine, so gradients flow from the verdict back into whatever produced the image.

Formulation. The non-saturating form from Goodfellow et al. (2014), which pix2pix (Isola et al., 2017) describes as training “G to maximize log D(x, G(x, z))” rather than minimizing log(1 - D) — the latter’s gradient vanishes exactly when the generator is losing, which is when it most needs signal. Concretely, with discriminator logits d:

generator: mean(softplus(-d_fake)); discriminator: mean(softplus(-d_real)) + mean(softplus(d_fake)), where softplus(x) = log(1 + e^x). This is binary cross-entropy with the sigmoid folded in, evaluated through the numerically stable identity softplus(x) = max(x, 0) + log(1 + e^-|x|) so no term ever overflows or takes the logarithm of zero.

Discriminator output convention. Prefer a discriminator that emits LOGITS (new PatchGANDiscriminator<T>(applySigmoid: false)). That is the same objective the paper describes — its sigmoid is folded into this loss — but computed stably. If the discriminator already applies a sigmoid, construct this loss with discriminatorOutputsProbabilities: true and the probabilities are clamped away from 0 and 1 before the logarithm.

Patch averaging. A PatchGAN emits a GRID of verdicts. Averaging that grid is exactly the paper’s “run convolutionally across the image, averaging all responses”, so no separate handling is needed: the reduction here is the averaging step.

Usage:

Constructors

Constructor Summary
AdversarialLoss(ILayer<T> discriminator, bool discriminatorOutputsProbabilities) Creates the adversarial loss.

Properties

Property Summary
ILayer<T> Discriminator { get; } Gets the discriminator, so a training loop can update it on its own alternating step and put it in the right training/evaluation mode.

Methods

Method Summary
T CalculateLoss(Vector<T> predicted, Vector<T> actual)
Tensor<T> ComputeDiscriminatorTapeLoss(Tensor<T> generated, Tensor<T> real) Computes the DISCRIMINATOR’s loss for one alternating step.
Tensor<T> ComputeTapeLoss(Tensor<T> predicted, Tensor<T> target)