AdversarialLoss<T>
Models & Types · AiDotNet.LossFunctions
Adversarial (GAN) loss driven by a real discriminator, computed on the autodiff tape.
Inheritance: LossFunctionBase<T> → AdversarialLoss<T>
For Beginners
a GAN trains two networks against each other. The generator produces an image;
the discriminator tries to tell generated images from real ones. This loss provides both halves of
that game: ComputeTapeLoss is what the GENERATOR minimizes (it wants to be judged
real), and ComputeDiscriminatorTapeLoss is what the DISCRIMINATOR minimizes (it wants
to judge correctly). They are trained in alternation.
How It Works
Why this class exists. A GAN term is only real if the discriminator’s forward pass is part of
the differentiated graph — the generator learns exclusively through the discriminator’s gradient.
A “GAN loss” that scores a discriminator output with plain arithmetic outside the tape produces a
plausible-looking number while sending the generator no signal at all, which is indistinguishable
from having no adversarial term. Every operation here goes through Engine, so gradients flow
from the verdict back into whatever produced the image.
Formulation. The non-saturating form from Goodfellow et al. (2014), which pix2pix (Isola et
al., 2017) describes as training “G to maximize log D(x, G(x, z))” rather than minimizing
log(1 - D) — the latter’s gradient vanishes exactly when the generator is losing, which is
when it most needs signal. Concretely, with discriminator logits d:
generator: mean(softplus(-d_fake)); discriminator:
mean(softplus(-d_real)) + mean(softplus(d_fake)), where softplus(x) = log(1 + e^x).
This is binary cross-entropy with the sigmoid folded in, evaluated through the numerically stable
identity softplus(x) = max(x, 0) + log(1 + e^-|x|) so no term ever overflows or takes the
logarithm of zero.
Discriminator output convention. Prefer a discriminator that emits LOGITS
(new PatchGANDiscriminator<T>(applySigmoid: false)). That is the same objective the paper
describes — its sigmoid is folded into this loss — but computed stably. If the discriminator already
applies a sigmoid, construct this loss with discriminatorOutputsProbabilities: true and the
probabilities are clamped away from 0 and 1 before the logarithm.
Patch averaging. A PatchGAN emits a GRID of verdicts. Averaging that grid is exactly the paper’s “run convolutionally across the image, averaging all responses”, so no separate handling is needed: the reduction here is the averaging step.
Usage:
Constructors
| Constructor | Summary |
|---|---|
AdversarialLoss(ILayer<T> discriminator, bool discriminatorOutputsProbabilities) |
Creates the adversarial loss. |
Properties
| Property | Summary |
|---|---|
ILayer<T> Discriminator { get; } |
Gets the discriminator, so a training loop can update it on its own alternating step and put it in the right training/evaluation mode. |
Methods
| Method | Summary |
|---|---|
T CalculateLoss(Vector<T> predicted, Vector<T> actual) |
|
Tensor<T> ComputeDiscriminatorTapeLoss(Tensor<T> generated, Tensor<T> real) |
Computes the DISCRIMINATOR’s loss for one alternating step. |
Tensor<T> ComputeTapeLoss(Tensor<T> predicted, Tensor<T> target) |