EntmoidActivation<T>
Models & Types · AiDotNet.NeuralNetworks.Tabular
Entmoid activation function for NODE architecture.
Inheritance: ActivationFunctionBase<T> → EntmoidActivation<T>
For Beginners
Entmoid is like a “smart sigmoid”:
- Regular sigmoid smoothly goes from 0 to 1
- Entmoid can produce exact zeros (sparse output)
- This helps the model focus on important features and ignore noise
The alpha parameter controls sparsity:
- alpha = 1.5 (default): moderately sparse
- alpha = 2: sparsemax (can produce exact zeros)
- alpha → 1: approaches softmax (no sparsity)
How It Works
Entmoid is a sparse, differentiable activation that generalizes softmax to produce sparse outputs. It’s the element-wise version of entmax (sparse attention). The function is defined as: entmoid(x) = max(0, (alpha * x + 1) / (2 * alpha))^(1/(alpha-1))
Constructors
| Constructor | Summary |
|---|---|
EntmoidActivation(double alpha) |
Initializes the entmoid activation. |
Properties
| Property | Summary |
|---|---|
bool SupportsGpuTraining { get; } |
Gets whether entmoid supports GPU-resident training. |
double Alpha { get; } |
Gets the alpha parameter controlling sparsity. |
Methods
| Method | Summary |
|---|---|
Matrix<T> Derivative(Vector<T> input) |
Computes the derivative of entmoid for a tensor. |
T Activate(T input) |
Applies the entmoid activation function to a tensor. |
T Derivative(T input) |
Computes the derivative of entmoid for a tensor. |
Tensor<T> Activate(Tensor<T> input) |
Applies the entmoid activation function to a tensor. |
Tensor<T> Derivative(Tensor<T> input) |
Computes the derivative of entmoid for a tensor. |
Vector<T> Activate(Vector<T> input) |
Applies the entmoid activation function to a tensor. |
void ForwardGpu(IDirectGpuBackend backend, IGpuBuffer input, IGpuBuffer output, int size) |
Applies the entmoid activation function on GPU. |