PReLUActivation<T>
Models & Types · AiDotNet.ActivationFunctions
Implements the Parametric Rectified Linear Unit (PReLU) activation function for neural networks.
Inheritance: ActivationFunctionBase<T> → PReLUActivation<T>
For Beginners
PReLU is an improved version of the popular ReLU activation function.
While ReLU completely blocks negative values (turning them to zero), PReLU allows a small portion of negative values to pass through, controlled by a parameter called “alpha”.
How PReLU works:
- For positive inputs (x > 0): PReLU returns the input unchanged (just like ReLU)
- For negative inputs (x = 0): PReLU returns alpha * x (a scaled-down version of the input)
The alpha parameter is typically a small positive number (default 0.01). This “leakiness” helps prevent a problem called “dying ReLU” where neurons can get stuck and stop learning.
PReLU can be thought of as a sloped line for negative inputs rather than a flat line at zero.
Example
using AiDotNet.ActivationFunctions;
var activation = new PReLUActivation<double>();
double y = activation.Activate(0.5);
Console.WriteLine($"PReLUActivation: f(0.5) = {y:F4}");
Constructors
| Constructor | Summary |
|---|---|
PReLUActivation(double alpha) |
Initializes a new instance of the PReLU activation function with the specified alpha value. |
Methods
| Method | Summary |
|---|---|
T Activate(T input) |
Applies PReLU to a tensor via engine primitives so the gradient tape records every step. |
T Derivative(T input) |
Calculates the derivative (gradient) of the PReLU function for a single input value. |
Tensor<T> Activate(Tensor<T> input) |
Applies PReLU to a tensor via engine primitives so the gradient tape records every step. |
void UpdateAlpha(T newAlpha) |
Updates the alpha parameter of the PReLU function. |