LogSoftmaxActivation<T>
Models & Types · AiDotNet.ActivationFunctions
Implements the LogSoftmax activation function for neural networks.
Inheritance: ActivationFunctionBase<T> → LogSoftmaxActivation<T>
For Beginners
The LogSoftmax function combines two operations:
- First, it applies the “softmax” function, which converts a vector of numbers into probabilities
(values between 0 and 1 that sum to 1).
- Then, it takes the natural logarithm of these probabilities.
This function is commonly used in the final layer of neural networks for classification problems, especially when combined with Negative Log-Likelihood loss. It helps with:
- Numerical stability (preventing extremely small or large numbers)
- Making the math work better during training
- Producing outputs that work well for calculating classification probabilities
Unlike most activation functions, LogSoftmax operates on vectors (collections of numbers) rather than individual values, because it needs to consider all outputs together to calculate probabilities.
Example
using AiDotNet.ActivationFunctions;
var activation = new LogSoftmaxActivation<double>();
double y = activation.Activate(0.5);
Console.WriteLine($"LogSoftmaxActivation: f(0.5) = {y:F4}");
Constructors
| Constructor | Summary |
|---|---|
LogSoftmaxActivation() |
Methods
| Method | Summary |
|---|---|
Matrix<T> Derivative(Vector<T> input) |
Calculates the derivative (Jacobian matrix) of the LogSoftmax function for a vector input. |
Tensor<T> Activate(Tensor<T> input) |
Applies LogSoftmax to a tensor via Engine.TensorLogSoftmax so the gradient tape records the op. |
Vector<T> Activate(Vector<T> input) |
Applies LogSoftmax to a tensor via Engine.TensorLogSoftmax so the gradient tape records the op. |