PReLUActivation<T>

Models & Types · AiDotNet.ActivationFunctions

Implements the Parametric Rectified Linear Unit (PReLU) activation function for neural networks.

Inheritance: ActivationFunctionBase<T> → PReLUActivation<T>

For Beginners

PReLU is an improved version of the popular ReLU activation function.

While ReLU completely blocks negative values (turning them to zero), PReLU allows a small portion of negative values to pass through, controlled by a parameter called “alpha”.

How PReLU works:

  • For positive inputs (x > 0): PReLU returns the input unchanged (just like ReLU)
  • For negative inputs (x = 0): PReLU returns alpha * x (a scaled-down version of the input)

The alpha parameter is typically a small positive number (default 0.01). This “leakiness” helps prevent a problem called “dying ReLU” where neurons can get stuck and stop learning.

PReLU can be thought of as a sloped line for negative inputs rather than a flat line at zero.

Example

using AiDotNet.ActivationFunctions;

var activation = new PReLUActivation<double>();
double y = activation.Activate(0.5);
Console.WriteLine($"PReLUActivation: f(0.5) = {y:F4}");

Constructors

Constructor Summary
PReLUActivation(double alpha) Initializes a new instance of the PReLU activation function with the specified alpha value.

Methods

Method Summary
T Activate(T input) Applies PReLU to a tensor via engine primitives so the gradient tape records every step.
T Derivative(T input) Calculates the derivative (gradient) of the PReLU function for a single input value.
Tensor<T> Activate(Tensor<T> input) Applies PReLU to a tensor via engine primitives so the gradient tape records every step.
void UpdateAlpha(T newAlpha) Updates the alpha parameter of the PReLU function.