LiSHTActivation<T>

Models & Types · AiDotNet.ActivationFunctions

Implements the Linearly Scaled Hyperbolic Tangent (LiSHT) activation function for neural networks.

Inheritance: ActivationFunctionBase<T> → LiSHTActivation<T>

Implements: IFusedActivation

For Beginners

The LiSHT activation function combines the input value with its hyperbolic tangent.

The formula is: f(x) = x * tanh(x)

This means:

  • For positive inputs: The output is positive but scaled down
  • For negative inputs: The output is negative but scaled down
  • For zero: The output is zero

LiSHT has several advantages:

  • It doesn’t suffer from the “vanishing gradient problem” (where learning becomes very slow)
  • It’s smooth everywhere (unlike ReLU which has a sharp corner at zero)
  • It naturally keeps values in a reasonable range

Think of it as a function that “squeezes” large values while preserving the sign and allowing small values to pass through with minimal change.

Example

using AiDotNet.ActivationFunctions;

var activation = new LiSHTActivation<double>();
double y = activation.Activate(0.5);
Console.WriteLine($"LiSHTActivation: f(0.5) = {y:F4}");

Constructors

Constructor Summary
LiSHTActivation()

Methods

Method Summary
T Activate(T input) Applies LiSHT to a tensor via engine primitives so the gradient tape records every step of x * tanh(x).
T Derivative(T input) Calculates the derivative of the LiSHT function for a single input value.
Tensor<T> Activate(Tensor<T> input) Applies LiSHT to a tensor via engine primitives so the gradient tape records every step of x * tanh(x).