CELUActivation<T>
Models & Types · AiDotNet.ActivationFunctions
Implements the Continuously Differentiable Exponential Linear Unit (CELU) activation function for neural networks.
Inheritance: ActivationFunctionBase<T> → CELUActivation<T>
For Beginners
The CELU activation function is an improved version of the popular ReLU function. While ReLU simply turns negative values to zero (which can cause “dead neurons”), CELU replaces negative values with a smooth exponential curve that approaches a negative limit.
Key benefits of CELU:
- For positive inputs, it behaves exactly like ReLU (returns the input value)
- For negative inputs, it returns a negative value that smoothly approaches -a
- This smooth transition helps prevent “dead neurons” during training
- The a parameter controls how quickly the function approaches its negative limit
CELU is particularly useful in deep neural networks where maintaining gradient flow through all neurons is important for effective learning.
Example
using AiDotNet.ActivationFunctions;
var activation = new CELUActivation<double>();
double y = activation.Activate(0.5);
Console.WriteLine($"CELUActivation: f(0.5) = {y:F4}");
Constructors
| Constructor | Summary |
|---|---|
CELUActivation(double alpha) |
Initializes a new instance of the CELUActivation class with the specified alpha parameter. |
Methods
| Method | Summary |
|---|---|
T Activate(T input) |
Applies CELU to a tensor via engine primitives so the gradient tape records every step. |
T Derivative(T input) |
Calculates the derivative of the CELU function for a single input value. |
Tensor<T> Activate(Tensor<T> input) |
Applies CELU to a tensor via engine primitives so the gradient tape records every step. |