ReLUActivation<T>

Models & Types · AiDotNet.ActivationFunctions

Implements the Rectified Linear Unit (ReLU) activation function.

Inheritance: ActivationFunctionBase<T> → ReLUActivation<T>

Implements: IFusedActivation

For Beginners

ReLU (Rectified Linear Unit) is like a filter that only lets positive values pass through unchanged, while changing all negative values to zero. Think of it as a function that “turns off” neurons that have negative values and keeps positive ones as they are. This helps neural networks learn more effectively by introducing non-linearity.

How It Works

The ReLU function is one of the most commonly used activation functions in neural networks. It outputs the input directly if it is positive, otherwise, it outputs zero.

Example

using AiDotNet.ActivationFunctions;

var activation = new ReLUActivation<double>();
double y = activation.Activate(0.5);
Console.WriteLine($"ReLUActivation: f(0.5) = {y:F4}");

Constructors

Constructor Summary
ReLUActivation()

Properties

Property Summary
bool SupportsGpuTraining { get; } Gets whether ReLU supports GPU-resident training.

Methods

Method Summary
Matrix<T> Derivative(Vector<T> input) Calculates the derivative of the ReLU function for each element in a tensor.
T Activate(T input) Applies the ReLU activation function to each element in a tensor.
T Derivative(T input) Calculates the derivative of the ReLU function for each element in a tensor.
Tensor<T> Activate(Tensor<T> input) Applies the ReLU activation function to each element in a tensor.
Tensor<T> Derivative(Tensor<T> input) Calculates the derivative of the ReLU function for each element in a tensor.
Vector<T> Activate(Vector<T> input) Applies the ReLU activation function to each element in a tensor.
void ForwardGpu(IDirectGpuBackend backend, IGpuBuffer input, IGpuBuffer output, int size) Applies the ReLU activation function on GPU.