IActivationFunction<T>

Interfaces · AiDotNet.Interfaces

Defines an interface for activation functions used in neural networks and other machine learning algorithms.

Known implementations: ActivationFunctionBase<T>

How It Works

For Beginners: An activation function is like a decision-maker in a neural network.

Imagine each neuron (node) in a neural network receives a number as input. The activation function decides how strongly that neuron should “fire” or activate based on that input.

For example:

  • If the input is very negative, the neuron might not activate at all (output = 0)
  • If the input is very positive, the neuron might activate fully (output = 1)
  • If the input is around zero, the neuron might activate partially

Different activation functions create different patterns of activation, which helps neural networks learn different types of patterns in data. Common activation functions include Sigmoid, ReLU (Rectified Linear Unit), and Tanh (Hyperbolic Tangent).

This interface defines the standard methods that all activation functions must implement.

Properties

Property Summary
bool SupportsGpuTraining { get; } Gets whether this activation function supports GPU-resident training.

Methods

Method Summary
Matrix<T> Derivative(Vector<T> input) Calculates the derivative for each element in a tensor.
T Activate(T input) Applies the activation function to each element in a tensor.
T Derivative(T input) Calculates the derivative for each element in a tensor.
Tensor<T> Activate(Tensor<T> input) Applies the activation function to each element in a tensor.
Tensor<T> Derivative(Tensor<T> input) Calculates the derivative for each element in a tensor.
Vector<T> Activate(Vector<T> input) Applies the activation function to each element in a tensor.
void ForwardGpu(IDirectGpuBackend backend, IGpuBuffer input, IGpuBuffer output, int size) Applies the activation function on GPU.