BentIdentityActivation<T>

Models & Types · AiDotNet.ActivationFunctions

Implements the Bent Identity activation function for neural networks.

Inheritance: ActivationFunctionBase<T> → BentIdentityActivation<T>

Implements: IFusedActivation

For Beginners

The Bent Identity activation function is a smoother alternative to the ReLU function. It behaves similarly to a linear function for positive inputs but has a gentle curve for negative inputs. This helps prevent the “dying neuron” problem that can occur with ReLU, where neurons can get stuck outputting zero.

The mathematical formula is: f(x) = ((sqrt(x² + 1) - 1) / 2) + x

Key properties:

  • Always produces a non-zero gradient, helping with training
  • Approximates linear behavior for large positive values
  • Provides a smooth transition around zero
  • Has no upper or lower bounds (unlike sigmoid or tanh)

Example

using AiDotNet.ActivationFunctions;

var activation = new BentIdentityActivation<double>();
double y = activation.Activate(0.5);
Console.WriteLine($"BentIdentityActivation: f(0.5) = {y:F4}");

Constructors

Constructor Summary
BentIdentityActivation()

Methods

Method Summary
T Activate(T input) Applies BentIdentity to a tensor via engine primitives so the gradient tape records every step of (sqrt(x² + 1) - 1) / 2 + x.
T Derivative(T input) Calculates the derivative of the Bent Identity function for a single input value.
Tensor<T> Activate(Tensor<T> input) Applies BentIdentity to a tensor via engine primitives so the gradient tape records every step of (sqrt(x² + 1) - 1) / 2 + x.