TanhActivation<T>
Models & Types · AiDotNet.ActivationFunctions
Implements the Hyperbolic Tangent (tanh) activation function for neural networks.
Inheritance: ActivationFunctionBase<T> → TanhActivation<T>
Implements: IFusedActivation, IOutputDerivative<T>
For Beginners
The Hyperbolic Tangent (tanh) activation function is a popular choice in neural networks. It transforms any input value to an output between -1 and 1, creating an S-shaped curve that’s symmetric around the origin.
Key properties of tanh:
- Outputs values between -1 and 1
- An input of 0 produces an output of 0
- Large positive inputs approach +1
- Large negative inputs approach -1
- It’s zero-centered, which often helps with learning
When to use tanh:
- When you need outputs centered around zero
- For hidden layers in many types of neural networks
- When dealing with data that naturally has both positive and negative values
One limitation is the “vanishing gradient problem” - for very large or small inputs, the function’s slope becomes very small, which can slow down learning in deep networks.
Example
using AiDotNet.ActivationFunctions;
var activation = new TanhActivation<double>();
double y = activation.Activate(0.5);
Console.WriteLine($"TanhActivation: f(0.5) = {y:F4}");
Constructors
| Constructor | Summary |
|---|---|
TanhActivation() |
Properties
| Property | Summary |
|---|---|
bool SupportsGpuTraining { get; } |
Gets whether Tanh supports GPU-resident training. |
Methods
| Method | Summary |
|---|---|
T Activate(T input) |
Applies the tanh activation function to a tensor of input values. |
T Derivative(T input) |
Calculates the derivative of the tanh function for a tensor input. |
Tensor<T> Activate(Tensor<T> input) |
Applies the tanh activation function to a tensor of input values. |
Tensor<T> Derivative(Tensor<T> input) |
Calculates the derivative of the tanh function for a tensor input. |
Tensor<T> DerivativeFromOutput(Tensor<T> output) |
Computes tanh derivative given the post-activation output: 1 - y². |
Vector<T> Activate(Vector<T> input) |
Applies the tanh activation function to a tensor of input values. |
void ForwardGpu(IDirectGpuBackend backend, IGpuBuffer input, IGpuBuffer output, int size) |
Applies the Tanh activation function on GPU. |