SoftPlusActivation<T>
Models & Types · AiDotNet.ActivationFunctions
Implements the SoftPlus activation function, which is a smooth approximation of the ReLU function.
Inheritance: ActivationFunctionBase<T> → SoftPlusActivation<T>
Implements: IFusedActivation
For Beginners
SoftPlus is like a “softer” version of the popular ReLU activation function. While ReLU outputs exactly 0 for any negative input and keeps positive values unchanged, SoftPlus creates a smooth curve that’s very close to ReLU but without the sharp corner at x=0.
For negative inputs, SoftPlus outputs small positive values (approaching 0). For large positive inputs, SoftPlus outputs values very close to the input itself.
This smoothness can be helpful in some neural networks because it means the function is differentiable everywhere (it has a well-defined slope at every point), which can make training more stable.
How It Works
The SoftPlus function is defined as: f(x) = ln(1 + e^x), where ln is the natural logarithm. It produces output that is always positive and approaches the ReLU function (max(0,x)) but with a smooth transition at x=0.
Example
using AiDotNet.ActivationFunctions;
var activation = new SoftPlusActivation<double>();
double y = activation.Activate(0.5);
Console.WriteLine($"SoftPlusActivation: f(0.5) = {y:F4}");
Constructors
| Constructor | Summary |
|---|---|
SoftPlusActivation() |
Properties
| Property | Summary |
|---|---|
bool SupportsGpuTraining { get; } |
Gets whether SoftPlus supports GPU-resident training. |
Methods
| Method | Summary |
|---|---|
T Activate(T input) |
Applies the SoftPlus activation function to each element in a tensor. |
T Derivative(T input) |
Calculates the derivative of the SoftPlus function for each element in a tensor. |
Tensor<T> Activate(Tensor<T> input) |
Applies the SoftPlus activation function to each element in a tensor. |
Tensor<T> Derivative(Tensor<T> input) |
Calculates the derivative of the SoftPlus function for each element in a tensor. |
void ForwardGpu(IDirectGpuBackend backend, IGpuBuffer input, IGpuBuffer output, int size) |
Applies the SoftPlus activation function on GPU. |