SoftmaxActivation<T>

Models & Types · AiDotNet.ActivationFunctions

Implements the Softmax activation function, which converts a vector of real numbers into a probability distribution.

Inheritance: ActivationFunctionBase<T> → SoftmaxActivation<T>

For Beginners

Softmax is commonly used in the output layer of neural networks for classification problems. Think of it as a way to convert raw scores (called “logits”) into probabilities. For example, if you’re classifying images into 3 categories (cat, dog, bird), the neural network might output raw scores like [2.5, 1.2, 0.8]. Softmax converts these to probabilities like [0.65, 0.22, 0.13], which sum to 1.0 (or 100%). This makes it easy to interpret the highest value as the model’s prediction (in this case, “cat” with 65% confidence).

Unlike other activation functions that work on single values, Softmax needs to see all values at once because it normalizes them relative to each other.

How It Works

The Softmax function takes a vector of real numbers and normalizes it into a probability distribution, where each value is between 0 and 1, and all values sum to 1. It’s defined as: softmax(x_i) = exp(x_i) / sum(exp(x_j)) for all j in the vector.

Example

using AiDotNet.ActivationFunctions;

var activation = new SoftmaxActivation<double>();
double y = activation.Activate(0.5);
Console.WriteLine($"SoftmaxActivation: f(0.5) = {y:F4}");

Constructors

Constructor Summary
SoftmaxActivation()

Properties

Property Summary
bool SupportsGpuTraining { get; } Gets whether Softmax supports GPU-resident training.

Methods

Method Summary
Matrix<T> Derivative(Vector<T> input) Calculates the Jacobian matrix of the Softmax function for a vector input.
Tensor<T> Activate(Tensor<T> input) Applies the Softmax activation function to each element in a tensor.
Vector<T> Activate(Vector<T> input) Applies the Softmax activation function to each element in a tensor.
void ForwardGpu(IDirectGpuBackend backend, IGpuBuffer input, IGpuBuffer output, int size) Applies the Softmax activation function on GPU.