TaylorSoftmaxActivation<T>

Models & Types · AiDotNet.ActivationFunctions

Implements the Taylor Softmax activation function, which is a computationally efficient approximation of the standard Softmax function.

Inheritance: ActivationFunctionBase<T> → TaylorSoftmaxActivation<T>

For Beginners

The Taylor Softmax function is a variation of the standard Softmax function that uses a mathematical technique called “Taylor series expansion” to approximate the exponential function. This makes it computationally more efficient while maintaining similar behavior to the standard Softmax.

Softmax functions are commonly used in the output layer of neural networks for multi-class classification problems. They convert a vector of numbers into a probability distribution (all values are positive and sum to 1).

For example, if you have three output neurons with values [2.0, 1.0, 0.5], the Softmax function will convert these to probabilities like [0.6, 0.25, 0.15], which sum to 1.0. This makes it easy to interpret the outputs as probabilities for each class.

The “Taylor” part refers to using a mathematical approximation (Taylor series) instead of calculating the full exponential function, which can be faster but slightly less accurate.

Example

using AiDotNet.ActivationFunctions;

var activation = new TaylorSoftmaxActivation<double>();
double y = activation.Activate(0.5);
Console.WriteLine($"TaylorSoftmaxActivation: f(0.5) = {y:F4}");

Constructors

Constructor Summary
TaylorSoftmaxActivation(int order) Initializes a new instance of the TaylorSoftmaxActivation class with the specified order of approximation.

Methods

Method Summary
Matrix<T> Derivative(Vector<T> input) Calculates the Jacobian matrix of partial derivatives for the Taylor Softmax function.
Tensor<T> Activate(Tensor<T> input) Applies TaylorSoftmax to a tensor via engine primitives so the gradient tape records every step.
Vector<T> Activate(Vector<T> input) Applies TaylorSoftmax to a tensor via engine primitives so the gradient tape records every step.