ReLUActivation<T>
Models & Types · AiDotNet.ActivationFunctions
Implements the Rectified Linear Unit (ReLU) activation function.
Inheritance: ActivationFunctionBase<T> → ReLUActivation<T>
Implements: IFusedActivation
For Beginners
ReLU (Rectified Linear Unit) is like a filter that only lets positive values pass through unchanged, while changing all negative values to zero. Think of it as a function that “turns off” neurons that have negative values and keeps positive ones as they are. This helps neural networks learn more effectively by introducing non-linearity.
How It Works
The ReLU function is one of the most commonly used activation functions in neural networks. It outputs the input directly if it is positive, otherwise, it outputs zero.
Example
using AiDotNet.ActivationFunctions;
var activation = new ReLUActivation<double>();
double y = activation.Activate(0.5);
Console.WriteLine($"ReLUActivation: f(0.5) = {y:F4}");
Constructors
| Constructor | Summary |
|---|---|
ReLUActivation() |
Properties
| Property | Summary |
|---|---|
bool SupportsGpuTraining { get; } |
Gets whether ReLU supports GPU-resident training. |
Methods
| Method | Summary |
|---|---|
Matrix<T> Derivative(Vector<T> input) |
Calculates the derivative of the ReLU function for each element in a tensor. |
T Activate(T input) |
Applies the ReLU activation function to each element in a tensor. |
T Derivative(T input) |
Calculates the derivative of the ReLU function for each element in a tensor. |
Tensor<T> Activate(Tensor<T> input) |
Applies the ReLU activation function to each element in a tensor. |
Tensor<T> Derivative(Tensor<T> input) |
Calculates the derivative of the ReLU function for each element in a tensor. |
Vector<T> Activate(Vector<T> input) |
Applies the ReLU activation function to each element in a tensor. |
void ForwardGpu(IDirectGpuBackend backend, IGpuBuffer input, IGpuBuffer output, int size) |
Applies the ReLU activation function on GPU. |