ScaledTanhActivation<T>

Models & Types · AiDotNet.ActivationFunctions

Implements the Scaled Hyperbolic Tangent (tanh) activation function for neural networks.

Inheritance: ActivationFunctionBase<T> → ScaledTanhActivation<T>

For Beginners

The Scaled Tanh activation function is a parameterized version of the standard hyperbolic tangent function. Like the standard tanh, it outputs values between -1 and 1, making it useful for neural networks where you want the output to be centered around zero.

The mathematical formula is: f(x) = (1 - e^(-ßx)) / (1 + e^(-ßx))

This is equivalent to the standard tanh function when ß = 2, and has these key properties:

  • Outputs values between -1 and 1
  • Is symmetric around the origin (f(-x) = -f(x))
  • The parameter ß (beta) controls the steepness of the curve
  • When ß = 2, this is exactly equivalent to the standard tanh function

When to use it:

  • When you need outputs centered around zero
  • For hidden layers in many types of neural networks
  • When dealing with data that naturally has both positive and negative values
  • When you want to control the steepness of the activation function

Example

using AiDotNet.ActivationFunctions;

var activation = new ScaledTanhActivation<double>();
double y = activation.Activate(0.5);
Console.WriteLine($"ScaledTanhActivation: f(0.5) = {y:F4}");

Constructors

Constructor Summary
ScaledTanhActivation(double beta, double saturationThreshold) Initializes a new instance of the ScaledTanhActivation class with the specified steepness parameter.

Methods

Method Summary
T Activate(T input) Applies ScaledTanh to a tensor via engine primitives so the gradient tape records every step.
T Derivative(T input) Calculates the derivative of the Scaled Tanh function for a single input value.
Tensor<T> Activate(Tensor<T> input) Applies ScaledTanh to a tensor via engine primitives so the gradient tape records every step.