LogSoftmaxActivation<T>

Models & Types · AiDotNet.ActivationFunctions

Implements the LogSoftmax activation function for neural networks.

Inheritance: ActivationFunctionBase<T> → LogSoftmaxActivation<T>

For Beginners

The LogSoftmax function combines two operations:

  1. First, it applies the “softmax” function, which converts a vector of numbers into probabilities

(values between 0 and 1 that sum to 1).

  1. Then, it takes the natural logarithm of these probabilities.

This function is commonly used in the final layer of neural networks for classification problems, especially when combined with Negative Log-Likelihood loss. It helps with:

  • Numerical stability (preventing extremely small or large numbers)
  • Making the math work better during training
  • Producing outputs that work well for calculating classification probabilities

Unlike most activation functions, LogSoftmax operates on vectors (collections of numbers) rather than individual values, because it needs to consider all outputs together to calculate probabilities.

Example

using AiDotNet.ActivationFunctions;

var activation = new LogSoftmaxActivation<double>();
double y = activation.Activate(0.5);
Console.WriteLine($"LogSoftmaxActivation: f(0.5) = {y:F4}");

Constructors

Constructor Summary
LogSoftmaxActivation()

Methods

Method Summary
Matrix<T> Derivative(Vector<T> input) Calculates the derivative (Jacobian matrix) of the LogSoftmax function for a vector input.
Tensor<T> Activate(Tensor<T> input) Applies LogSoftmax to a tensor via Engine.TensorLogSoftmax so the gradient tape records the op.
Vector<T> Activate(Vector<T> input) Applies LogSoftmax to a tensor via Engine.TensorLogSoftmax so the gradient tape records the op.