EntmoidActivation<T>

Models & Types · AiDotNet.NeuralNetworks.Tabular

Entmoid activation function for NODE architecture.

Inheritance: ActivationFunctionBase<T> → EntmoidActivation<T>

For Beginners

Entmoid is like a “smart sigmoid”:

  • Regular sigmoid smoothly goes from 0 to 1
  • Entmoid can produce exact zeros (sparse output)
  • This helps the model focus on important features and ignore noise

The alpha parameter controls sparsity:

  • alpha = 1.5 (default): moderately sparse
  • alpha = 2: sparsemax (can produce exact zeros)
  • alpha → 1: approaches softmax (no sparsity)

How It Works

Entmoid is a sparse, differentiable activation that generalizes softmax to produce sparse outputs. It’s the element-wise version of entmax (sparse attention). The function is defined as: entmoid(x) = max(0, (alpha * x + 1) / (2 * alpha))^(1/(alpha-1))

Constructors

Constructor Summary
EntmoidActivation(double alpha) Initializes the entmoid activation.

Properties

Property Summary
bool SupportsGpuTraining { get; } Gets whether entmoid supports GPU-resident training.
double Alpha { get; } Gets the alpha parameter controlling sparsity.

Methods

Method Summary
Matrix<T> Derivative(Vector<T> input) Computes the derivative of entmoid for a tensor.
T Activate(T input) Applies the entmoid activation function to a tensor.
T Derivative(T input) Computes the derivative of entmoid for a tensor.
Tensor<T> Activate(Tensor<T> input) Applies the entmoid activation function to a tensor.
Tensor<T> Derivative(Tensor<T> input) Computes the derivative of entmoid for a tensor.
Vector<T> Activate(Vector<T> input) Applies the entmoid activation function to a tensor.
void ForwardGpu(IDirectGpuBackend backend, IGpuBuffer input, IGpuBuffer output, int size) Applies the entmoid activation function on GPU.