ELUActivation<T>

Models & Types · AiDotNet.ActivationFunctions

Implements the Exponential Linear Unit (ELU) activation function for neural networks.

Inheritance: ActivationFunctionBase<T> → ELUActivation<T>

For Beginners

ELU is an activation function that, like ReLU, returns the input directly for positive values. For negative inputs, it returns a smooth curve that approaches -alpha as the input becomes more negative.

Key advantages of ELU include:

  • It helps prevent “dying neurons” (a problem with ReLU) by allowing negative values
  • It has a smooth curve for negative inputs, which can help with gradient-based learning
  • The parameter alpha controls how negative the curve can go
  • It centers the activations closer to zero, which can speed up learning

ELU is often used in deep neural networks where ReLU might cause training issues.

Example

using AiDotNet.ActivationFunctions;

var activation = new ELUActivation<double>();
double y = activation.Activate(0.5);
Console.WriteLine($"ELUActivation: f(0.5) = {y:F4}");

Constructors

Constructor Summary
ELUActivation(double alpha) Initializes a new instance of the ELUActivation class.

Properties

Property Summary
T Alpha { get; } Gets the alpha parameter that controls the saturation value for negative inputs.
bool SupportsGpuTraining { get; } Gets whether ELU supports GPU-resident training.

Methods

Method Summary
Matrix<T> Derivative(Vector<T> input) Calculates the derivative of the ELU activation function for each element of an input vector.
T Activate(T x) Applies the ELU activation function to each element of an input vector.
T Derivative(T x) Calculates the derivative of the ELU activation function for each element of an input vector.
Tensor<T> Activate(Tensor<T> input) Applies the ELU activation function to each element of an input vector.
Vector<T> Activate(Vector<T> input) Applies the ELU activation function to each element of an input vector.
void ForwardGpu(IDirectGpuBackend backend, IGpuBuffer input, IGpuBuffer output, int size) Applies the ELU activation function on GPU.