HardSigmoidActivation<T>

Models & Types ยท AiDotNet.ActivationFunctions

Implements the Hard Sigmoid activation function for neural networks.

Inheritance: ActivationFunctionBase<T> โ†’ HardSigmoidActivation<T>

For Beginners

The Hard Sigmoid is a simplified version of the standard Sigmoid function.

While the regular Sigmoid creates an S-shaped curve that smoothly transitions from 0 to 1, the Hard Sigmoid uses straight lines to approximate this curve, making it:

  1. Computationally faster (uses simple math operations instead of exponentials)
  2. Less smooth but still useful for many neural network applications

The function works like this:

  • If input = -1: output = 0
  • If input = 1: output = 1
  • If -1 < input < 1: output = (input + 1) / 2

This creates a straight line between (-1, 0) and (1, 1), with values clamped to the range [0, 1].

Hard Sigmoid is often used in mobile or embedded applications where computational efficiency is important, or in certain types of recurrent neural networks.

Example

using AiDotNet.ActivationFunctions;

var activation = new HardSigmoidActivation<double>();
double y = activation.Activate(0.5);
Console.WriteLine($"HardSigmoidActivation: f(0.5) = {y:F4}");

Constructors

Constructor Summary
HardSigmoidActivation()

Properties

Property Summary
bool SupportsGpuTraining { get; } Gets whether HardSigmoid supports GPU-resident training.

Methods

Method Summary
T Activate(T input) Applies HardSigmoid to a tensor via engine primitives so the gradient tape records every step.
T Derivative(T input) Calculates the derivative of the Hard Sigmoid function for a given input value.
Tensor<T> Activate(Tensor<T> input) Applies HardSigmoid to a tensor via engine primitives so the gradient tape records every step.
void ForwardGpu(IDirectGpuBackend backend, IGpuBuffer input, IGpuBuffer output, int size) Applies the HardSigmoid activation function on GPU.