CrossEntropyWithLogitsLoss<T>

Models & Types · AiDotNet.LossFunctions

Implements Cross-Entropy loss that accepts raw logits (not probabilities).

Inheritance: LossFunctionBase<T> → CrossEntropyWithLogitsLoss<T>

For Beginners

This loss function is equivalent to PyTorch’s nn.CrossEntropyLoss. It combines LogSoftmax and Negative Log-Likelihood in a single numerically stable computation.

Unlike CrossEntropyLoss which expects probability inputs (after softmax), this version accepts raw logits (unbounded model outputs) and applies the softmax internally. This is the correct choice when your model’s final layer outputs raw scores without softmax activation.

The formula uses the log-sum-exp trick for numerical stability: loss = -logit[target_class] + log(sum(exp(logit_i)))

For soft targets (one-hot encoded): loss = -sum(target_i * (logit_i - log(sum(exp(logit_j)))))

Example

using AiDotNet.LossFunctions;
using AiDotNet.Tensors.LinearAlgebra;

var loss = new CrossEntropyWithLogitsLoss<float>();
var predicted = new Vector<float>(new[] { 0.9f, 0.2f, 0.7f });
var actual = new Vector<float>(new[] { 1.0f, 0.0f, 1.0f });

float value = loss.CalculateLoss(predicted, actual);
Console.WriteLine($"CrossEntropyWithLogitsLoss = {value:F4}");

Constructors

Constructor Summary
CrossEntropyWithLogitsLoss() Initializes cross-entropy with automatic class-axis inference.
CrossEntropyWithLogitsLoss(int classAxis) Initializes cross-entropy with an explicit class axis.

Properties

Property Summary
int? ClassAxis { get; } Gets the explicitly configured class axis, or null when inference is enabled.

Methods

Method Summary
T CalculateLoss(Vector<T> predicted, Vector<T> actual) Calculates the Cross-Entropy loss from raw logits using log-sum-exp for stability.
Tensor<T> ComputeTapeLoss(Tensor<T> predicted, Tensor<T> target)