BentIdentityActivation<T>
Models & Types · AiDotNet.ActivationFunctions
Implements the Bent Identity activation function for neural networks.
Inheritance: ActivationFunctionBase<T> → BentIdentityActivation<T>
Implements: IFusedActivation
For Beginners
The Bent Identity activation function is a smoother alternative to the ReLU function. It behaves similarly to a linear function for positive inputs but has a gentle curve for negative inputs. This helps prevent the “dying neuron” problem that can occur with ReLU, where neurons can get stuck outputting zero.
The mathematical formula is: f(x) = ((sqrt(x² + 1) - 1) / 2) + x
Key properties:
- Always produces a non-zero gradient, helping with training
- Approximates linear behavior for large positive values
- Provides a smooth transition around zero
- Has no upper or lower bounds (unlike sigmoid or tanh)
Example
using AiDotNet.ActivationFunctions;
var activation = new BentIdentityActivation<double>();
double y = activation.Activate(0.5);
Console.WriteLine($"BentIdentityActivation: f(0.5) = {y:F4}");
Constructors
| Constructor | Summary |
|---|---|
BentIdentityActivation() |
Methods
| Method | Summary |
|---|---|
T Activate(T input) |
Applies BentIdentity to a tensor via engine primitives so the gradient tape records every step of (sqrt(x² + 1) - 1) / 2 + x. |
T Derivative(T input) |
Calculates the derivative of the Bent Identity function for a single input value. |
Tensor<T> Activate(Tensor<T> input) |
Applies BentIdentity to a tensor via engine primitives so the gradient tape records every step of (sqrt(x² + 1) - 1) / 2 + x. |