SELUActivation<T>
Models & Types · AiDotNet.ActivationFunctions
Implements the Scaled Exponential Linear Unit (SELU) activation function.
Inheritance: ActivationFunctionBase<T> → SELUActivation<T>
Implements: IFusedActivation
For Beginners
SELU (Scaled Exponential Linear Unit) is a special activation function that helps neural networks train more effectively. Unlike simpler functions like ReLU, SELU has carefully chosen constants (alpha and lambda) that help keep the data flowing through your neural network well-balanced. This means your network can learn faster and more reliably without requiring extra normalization steps. Think of it as a self-regulating function that keeps your data in a “sweet spot” range as it passes through the network.
How It Works
SELU is an activation function that enables self-normalizing properties in neural networks. It automatically ensures that the outputs maintain a mean of 0 and standard deviation of 1 across the network, which helps with training stability and convergence.
Example
using AiDotNet.ActivationFunctions;
var activation = new SELUActivation<double>();
double y = activation.Activate(0.5);
Console.WriteLine($"SELUActivation: f(0.5) = {y:F4}");
Constructors
| Constructor | Summary |
|---|---|
SELUActivation() |
Initializes a new instance of the SELU activation function with predefined constants. |
Properties
| Property | Summary |
|---|---|
bool SupportsGpuTraining { get; } |
Gets whether SELU supports GPU-resident training. |
Methods
| Method | Summary |
|---|---|
T Activate(T input) |
Applies SELU to a tensor via the engine so the gradient tape records the op. |
T Derivative(T input) |
Calculates the derivative of the SELU function for a single input value. |
Tensor<T> Activate(Tensor<T> input) |
Applies SELU to a tensor via the engine so the gradient tape records the op. |
void ForwardGpu(IDirectGpuBackend backend, IGpuBuffer input, IGpuBuffer output, int size) |
Applies the SELU activation function on GPU. |