ELUActivation<T>
Models & Types · AiDotNet.ActivationFunctions
Implements the Exponential Linear Unit (ELU) activation function for neural networks.
Inheritance: ActivationFunctionBase<T> → ELUActivation<T>
For Beginners
ELU is an activation function that, like ReLU, returns the input directly for positive values. For negative inputs, it returns a smooth curve that approaches -alpha as the input becomes more negative.
Key advantages of ELU include:
- It helps prevent “dying neurons” (a problem with ReLU) by allowing negative values
- It has a smooth curve for negative inputs, which can help with gradient-based learning
- The parameter alpha controls how negative the curve can go
- It centers the activations closer to zero, which can speed up learning
ELU is often used in deep neural networks where ReLU might cause training issues.
Example
using AiDotNet.ActivationFunctions;
var activation = new ELUActivation<double>();
double y = activation.Activate(0.5);
Console.WriteLine($"ELUActivation: f(0.5) = {y:F4}");
Constructors
| Constructor | Summary |
|---|---|
ELUActivation(double alpha) |
Initializes a new instance of the ELUActivation class. |
Properties
| Property | Summary |
|---|---|
T Alpha { get; } |
Gets the alpha parameter that controls the saturation value for negative inputs. |
bool SupportsGpuTraining { get; } |
Gets whether ELU supports GPU-resident training. |
Methods
| Method | Summary |
|---|---|
Matrix<T> Derivative(Vector<T> input) |
Calculates the derivative of the ELU activation function for each element of an input vector. |
T Activate(T x) |
Applies the ELU activation function to each element of an input vector. |
T Derivative(T x) |
Calculates the derivative of the ELU activation function for each element of an input vector. |
Tensor<T> Activate(Tensor<T> input) |
Applies the ELU activation function to each element of an input vector. |
Vector<T> Activate(Vector<T> input) |
Applies the ELU activation function to each element of an input vector. |
void ForwardGpu(IDirectGpuBackend backend, IGpuBuffer input, IGpuBuffer output, int size) |
Applies the ELU activation function on GPU. |