SparsemaxActivation<T>

Models & Types · AiDotNet.ActivationFunctions

Implements the Sparsemax activation function, which is an alternative to Softmax that can produce sparse probability distributions.

Inheritance: ActivationFunctionBase<T> → SparsemaxActivation<T>

For Beginners

Sparsemax is an advanced activation function used primarily in the output layer of neural networks for classification tasks. While Softmax always gives some probability to every possible class (even if very small), Sparsemax can assign exactly zero probability to unlikely classes.

Think of it like this:

  • Softmax is like saying “I’m 80% sure it’s a dog, 15% sure it’s a cat, and 5% sure it’s something else”
  • Sparsemax might say “I’m 90% sure it’s a dog, 10% sure it’s a cat, and 0% sure it’s anything else”

This “sparsity” (having many zeros) can be useful when you have many possible classes but only a few are likely to be correct. It makes the model’s predictions more focused and interpretable.

How It Works

Sparsemax maps input vectors to probability distributions, similar to Softmax, but with the key difference that Sparsemax can assign exactly zero probability to low-scoring classes, creating sparse outputs.

Example

using AiDotNet.ActivationFunctions;

var activation = new SparsemaxActivation<double>();
double y = activation.Activate(0.5);
Console.WriteLine($"SparsemaxActivation: f(0.5) = {y:F4}");

Constructors

Constructor Summary
SparsemaxActivation()

Methods

Method Summary
Matrix<T> Derivative(Vector<T> input) Calculates the Jacobian matrix of the Sparsemax function for a given input vector.
Tensor<T> Activate(Tensor<T> input) Applies Sparsemax to a tensor via Engine.Sparsemax so the gradient tape records the op.
Vector<T> Activate(Vector<T> input) Applies Sparsemax to a tensor via Engine.Sparsemax so the gradient tape records the op.