CELUActivation<T>

Models & Types · AiDotNet.ActivationFunctions

Implements the Continuously Differentiable Exponential Linear Unit (CELU) activation function for neural networks.

Inheritance: ActivationFunctionBase<T> → CELUActivation<T>

For Beginners

The CELU activation function is an improved version of the popular ReLU function. While ReLU simply turns negative values to zero (which can cause “dead neurons”), CELU replaces negative values with a smooth exponential curve that approaches a negative limit.

Key benefits of CELU:

  • For positive inputs, it behaves exactly like ReLU (returns the input value)
  • For negative inputs, it returns a negative value that smoothly approaches -a
  • This smooth transition helps prevent “dead neurons” during training
  • The a parameter controls how quickly the function approaches its negative limit

CELU is particularly useful in deep neural networks where maintaining gradient flow through all neurons is important for effective learning.

Example

using AiDotNet.ActivationFunctions;

var activation = new CELUActivation<double>();
double y = activation.Activate(0.5);
Console.WriteLine($"CELUActivation: f(0.5) = {y:F4}");

Constructors

Constructor Summary
CELUActivation(double alpha) Initializes a new instance of the CELUActivation class with the specified alpha parameter.

Methods

Method Summary
T Activate(T input) Applies CELU to a tensor via engine primitives so the gradient tape records every step.
T Derivative(T input) Calculates the derivative of the CELU function for a single input value.
Tensor<T> Activate(Tensor<T> input) Applies CELU to a tensor via engine primitives so the gradient tape records every step.