AttentionKernelPath
Enums · AiDotNet.Enums
Identifies which attention kernel a layer’s most recent forward pass actually executed.
For Beginners
Attention can be computed two ways that give the same answer. One of them also hands back a big table of “how much each token looked at each other token”; that table is useful for diagnostics but costs a lot of memory on long sequences. This enum records which of the two was used last.
How It Works
The same attention block can be computed by more than one kernel, and the choice is a
memory decision rather than a mathematical one: the fused FlashAttention kernel never
returns the [batch, heads, seq_q, seq_kv] score matrix, so a forward that needs
those scores (the auxiliary losses, or a manual tape-free backward) has to take the
score-materializing kernel instead.
Reporting the path a forward took makes that decision observable, so a regression that silently reverts a layer to the expensive kernel is a failing assertion rather than a quiet slowdown (#2312).
Fields
| Field | Summary |
|---|---|
AttentionKernelPath AdditiveBias |
The additive-bias path used when a layer injects an ALiBi positional bias. |
AttentionKernelPath FusedFlashAttention |
The fused FlashAttention kernel (IEngine.FlashAttention), which keeps one log-sum-exp per query row instead of the full score matrix and masks causally in-kernel. |
AttentionKernelPath FusedMultiHeadCore |
The head-interleaved fused kernel (IEngine.MultiHeadAttentionCore), which reads the projected Q/K/V in their [batch, seq, heads*headDim] layout and returns the merged-head context, so the head-split and head-merge permutes are part of t… |
AttentionKernelPath None |
No attention forward has run yet on this layer. |
AttentionKernelPath ScoreMaterializing |
The score-materializing kernel (IEngine.ScaledDotProductAttention), which returns the attention weights because something downstream reads them. |
int value__ |