AttentionVisualizationExplainer<T>
Models & Types · AiDotNet.Interpretability.Explainers
Attention Visualization explainer for Transformer models.
Implements: ILocalExplainer<T, AttentionExplanation<T>>
For Beginners
Transformers use “attention” to decide which parts of the input to focus on when making predictions. This explainer visualizes these attention patterns.
What is attention?
- Each position in the input “attends” to all other positions
- Attention weights show how much each position influences another
- Higher weight = more influence/importance
Types of attention in Transformers:
- Self-attention: Input attends to itself (e.g., words attending to other words)
- Cross-attention: One sequence attends to another (e.g., decoder attending to encoder)
Visualization methods:
- Raw attention: Direct attention weights from the model
- Attention rollout: Combines attention across all layers
- Attention flow: Tracks information flow through the network
Example use cases:
- NLP: See which words the model focuses on for classification
- Vision Transformers: See which image patches are important
- Time Series: See which past timesteps influence predictions
Multi-head attention:
- Transformers have multiple “heads” that attend to different aspects
- This explainer can show individual heads or their combination
Constructors
| Constructor | Summary |
|---|---|
AttentionVisualizationExplainer(Func<Tensor<T>, Tensor<T>> predictFunction, Func<Tensor<T>, int, Tensor<T>> getAttentionWeights, int numLayers, int numHeads, int sequenceLength, string[] tokenLabels) |
Initializes a new Attention Visualization explainer. |
Properties
| Property | Summary |
|---|---|
bool SupportsGlobalExplanations { get; } |
|
bool SupportsLocalExplanations { get; } |
|
string MethodName { get; } |
Methods
| Method | Summary |
|---|---|
AttentionExplanation<T> Explain(Vector<T> instance) |
Visualizes attention patterns for an input. |
AttentionExplanation<T> ExplainTensor(Tensor<T> input) |
Visualizes attention patterns for an input tensor. |
AttentionExplanation<T>[] ExplainBatch(Matrix<T> instances) |
|
Dictionary<int, T> GetAttentionFromPosition(AttentionExplanation<T> explanation, int queryPosition, int layer, int head) |
Gets attention for a specific token/position. |
List<ValueTuple<int, string, T>> GetTopAttendedPositions(AttentionExplanation<T> explanation, int queryPosition, int topK, int layer) |
Gets the most attended positions for a query position. |