HardTanhActivation<T>

Models & Types · AiDotNet.ActivationFunctions

Implements the Hard Tanh activation function for neural networks.

Inheritance: ActivationFunctionBase<T> → HardTanhActivation<T>

For Beginners

The Hard Tanh is a simplified version of the standard Tanh (hyperbolic tangent) function.

While the regular Tanh creates an S-shaped curve that smoothly transitions from -1 to 1, the Hard Tanh uses straight lines to approximate this curve, making it:

  1. Computationally faster (uses simple comparison operations instead of complex math)
  2. Less smooth but still useful for many neural network applications

The function works like this:

  • If input = -1: output = -1
  • If input = 1: output = 1
  • If -1 < input < 1: output = input (unchanged)

This creates a function that “clips” or “saturates” any input to the range [-1, 1], with a straight line in between.

Hard Tanh is often used when you want the benefits of Tanh (centered around zero, outputs between -1 and 1) but need faster computation, such as in deep networks or resource-constrained environments.

Example

using AiDotNet.ActivationFunctions;

var activation = new HardTanhActivation<double>();
double y = activation.Activate(0.5);
Console.WriteLine($"HardTanhActivation: f(0.5) = {y:F4}");

Constructors

Constructor Summary
HardTanhActivation()

Properties

Property Summary
bool SupportsGpuTraining { get; } Gets whether HardTanh supports GPU-resident training.

Methods

Method Summary
T Activate(T input) Applies HardTanh to a tensor via engine primitives so the gradient tape records every step.
T Derivative(T input) Calculates the derivative of the Hard Tanh function for a given input value.
Tensor<T> Activate(Tensor<T> input) Applies HardTanh to a tensor via engine primitives so the gradient tape records every step.
void ForwardGpu(IDirectGpuBackend backend, IGpuBuffer input, IGpuBuffer output, int size) Applies the HardTanh activation function on GPU.