CrossEntropyWithLogitsLoss<T>
Models & Types · AiDotNet.LossFunctions
Implements Cross-Entropy loss that accepts raw logits (not probabilities).
Inheritance: LossFunctionBase<T> → CrossEntropyWithLogitsLoss<T>
For Beginners
This loss function is equivalent to PyTorch’s
nn.CrossEntropyLoss. It combines LogSoftmax and Negative Log-Likelihood
in a single numerically stable computation.
Unlike CrossEntropyLoss which expects probability inputs
(after softmax), this version accepts raw logits (unbounded model outputs)
and applies the softmax internally. This is the correct choice when your
model’s final layer outputs raw scores without softmax activation.
The formula uses the log-sum-exp trick for numerical stability: loss = -logit[target_class] + log(sum(exp(logit_i)))
For soft targets (one-hot encoded): loss = -sum(target_i * (logit_i - log(sum(exp(logit_j)))))
Example
using AiDotNet.LossFunctions;
using AiDotNet.Tensors.LinearAlgebra;
var loss = new CrossEntropyWithLogitsLoss<float>();
var predicted = new Vector<float>(new[] { 0.9f, 0.2f, 0.7f });
var actual = new Vector<float>(new[] { 1.0f, 0.0f, 1.0f });
float value = loss.CalculateLoss(predicted, actual);
Console.WriteLine($"CrossEntropyWithLogitsLoss = {value:F4}");
Constructors
| Constructor | Summary |
|---|---|
CrossEntropyWithLogitsLoss() |
Initializes cross-entropy with automatic class-axis inference. |
CrossEntropyWithLogitsLoss(int classAxis) |
Initializes cross-entropy with an explicit class axis. |
Properties
| Property | Summary |
|---|---|
int? ClassAxis { get; } |
Gets the explicitly configured class axis, or null when inference is enabled. |
Methods
| Method | Summary |
|---|---|
T CalculateLoss(Vector<T> predicted, Vector<T> actual) |
Calculates the Cross-Entropy loss from raw logits using log-sum-exp for stability. |
Tensor<T> ComputeTapeLoss(Tensor<T> predicted, Tensor<T> target) |