LiSHTActivation<T>
Models & Types · AiDotNet.ActivationFunctions
Implements the Linearly Scaled Hyperbolic Tangent (LiSHT) activation function for neural networks.
Inheritance: ActivationFunctionBase<T> → LiSHTActivation<T>
Implements: IFusedActivation
For Beginners
The LiSHT activation function combines the input value with its hyperbolic tangent.
The formula is: f(x) = x * tanh(x)
This means:
- For positive inputs: The output is positive but scaled down
- For negative inputs: The output is negative but scaled down
- For zero: The output is zero
LiSHT has several advantages:
- It doesn’t suffer from the “vanishing gradient problem” (where learning becomes very slow)
- It’s smooth everywhere (unlike ReLU which has a sharp corner at zero)
- It naturally keeps values in a reasonable range
Think of it as a function that “squeezes” large values while preserving the sign and allowing small values to pass through with minimal change.
Example
using AiDotNet.ActivationFunctions;
var activation = new LiSHTActivation<double>();
double y = activation.Activate(0.5);
Console.WriteLine($"LiSHTActivation: f(0.5) = {y:F4}");
Constructors
| Constructor | Summary |
|---|---|
LiSHTActivation() |
Methods
| Method | Summary |
|---|---|
T Activate(T input) |
Applies LiSHT to a tensor via engine primitives so the gradient tape records every step of x * tanh(x). |
T Derivative(T input) |
Calculates the derivative of the LiSHT function for a single input value. |
Tensor<T> Activate(Tensor<T> input) |
Applies LiSHT to a tensor via engine primitives so the gradient tape records every step of x * tanh(x). |