SigmoidActivation<T>

Models & Types · AiDotNet.ActivationFunctions

Implements the Sigmoid activation function, one of the most common activation functions in neural networks.

Inheritance: ActivationFunctionBase<T> → SigmoidActivation<T>

Implements: IFusedActivation, IOutputDerivative<T>

For Beginners

The Sigmoid function is like a “squashing” function that takes any number (from negative infinity to positive infinity) and converts it to a value between 0 and 1. This is useful in neural networks because it helps transform unbounded values into a probability-like range. The function creates an S-shaped curve that approaches 0 for very negative inputs and approaches 1 for very positive inputs, with a smooth transition in between. However, one limitation is that for extreme values (very large positive or negative), the gradient becomes very small, which can slow down learning in deep networks (known as the “vanishing gradient problem”).

How It Works

The Sigmoid function maps any input value to an output between 0 and 1, creating an S-shaped curve. It’s often used in the output layer of binary classification problems or in hidden layers of neural networks.

Example

using AiDotNet.ActivationFunctions;

var activation = new SigmoidActivation<double>();
double y = activation.Activate(0.5);
Console.WriteLine($"SigmoidActivation: f(0.5) = {y:F4}");

Constructors

Constructor Summary
SigmoidActivation()

Properties

Property Summary
bool SupportsGpuTraining { get; } Gets whether Sigmoid supports GPU-resident training.

Methods

Method Summary
Matrix<T> Derivative(Vector<T> input) Calculates the derivative of the Sigmoid function for each element in a tensor.
T Activate(T input) Applies the Sigmoid activation function to each element in a tensor.
T Derivative(T input) Calculates the derivative of the Sigmoid function for each element in a tensor.
Tensor<T> Activate(Tensor<T> input) Applies the Sigmoid activation function to each element in a tensor.
Tensor<T> Derivative(Tensor<T> input) Calculates the derivative of the Sigmoid function for each element in a tensor.
Tensor<T> DerivativeFromOutput(Tensor<T> output) Computes sigmoid derivative given the post-activation output: y * (1 - y).
Vector<T> Activate(Vector<T> input) Applies the Sigmoid activation function to each element in a tensor.
void ForwardGpu(IDirectGpuBackend backend, IGpuBuffer input, IGpuBuffer output, int size) Applies the Sigmoid activation function on GPU.