EntmaxAttention<T>
Models & Types · AiDotNet.NeuralNetworks.Tabular
Entmax sparse attention function for NODE architecture.
For Beginners
Entmax is like softmax but can completely ignore some inputs:
- Softmax: Every input gets some attention (even if tiny)
- Entmax: Unimportant inputs get exactly zero attention
This helps the model focus better and makes the attention patterns easier to interpret.
How It Works
Entmax is a sparse alternative to softmax that can produce exact zeros in the output distribution. This is useful for attention mechanisms where you want to focus on only a few important elements.
Constructors
| Constructor | Summary |
|---|---|
EntmaxAttention(double alpha) |
Initializes entmax attention. |
Methods
| Method | Summary |
|---|---|
Tensor<T> Forward(Tensor<T> scores) |
Applies entmax to convert scores to a sparse probability distribution. |