McnetLoss<T>

Helpers & Utilities · AiDotNet.Video.Prediction

MCnet’s objective: a pixel loss plus a GRADIENT DIFFERENCE loss in image space, combined with a conditional adversarial term.

For Beginners

Predicting the next video frame by minimising average pixel error gives a blurry compromise between everything that might happen. Also matching how sharply brightness CHANGES across the image — its gradients — forces crisp edges, and a critic network judging whether the continuation looks real given what came before pushes it further.

How It Works

Villegas, Yang, Hong, Lin and Lee, “Decomposing Motion and Content for Natural Video Sequence Prediction” (ICLR 2017, arXiv:1706.08033).

The gradient difference loss is the term a reimplementation drops. It penalises differences between the IMAGE GRADIENTS of prediction and target, not the pixels — which is what keeps predictions sharp. Omitting it leaves a plain L2, and L2 alone produces exactly the blur the paper exists to fight, because averaging over plausible futures minimises squared error.

The adversarial form is the NON-saturating one, -log D(G(.)), unlike FIGAN’s literal log(1 - D(G(.))). They are different objectives and should not be harmonised into one shared helper just because both models are adversarial.

The discriminator is CONDITIONED on the observed frames: it judges [x_1:t, future], never the future alone. An unconditional critic cannot tell a plausible continuation of THIS sequence from a plausible video in general, which is a weaker and different requirement.

Methods

Method Summary
double DiscriminatorLoss(double discriminatorOnReal, double discriminatorOnGenerated) L_disc = -log D([x, real]) - log(1 - D([x, fake])), which the discriminator MINIMISES.
double GeneratorAdversarial(double discriminatorOnGenerated) L_GAN = -log D([x_1:t, G(x_1:t)]), the generator’s NON-saturating adversarial term.
double GradientDifference(Vector<T> predicted, Vector<T> target, int height, int width, double lambda) L_gdl: the gradient difference loss over a [height, width] single-channel image.
double Image(Vector<T> predicted, Vector<T> target, int height, int width) L_img = L_p + L_gdl: the image-space loss, pixel plus gradient difference.
double Pixel(Vector<T> predicted, Vector<T> target, int p) L_p: the pixel loss, sum ||y - z||_p^p with p = 2 by default.
double Total(double imageLoss, double adversarialLoss, double adversarialWeight) L = alpha * L_img + beta * L_GAN.

Fields

Field Summary
double AdversarialWeightKth The paper’s adversarial weight beta for KTH.
double AdversarialWeightUcf The paper’s adversarial weight beta for UCF-101.
double GradientExponent The paper’s gradient-difference exponent lambda.
double ImageWeight The paper’s image-loss weight alpha.
int PixelNorm The paper’s pixel-loss norm p.