AttentionVisualizationExplainer<T>

Models & Types · AiDotNet.Interpretability.Explainers

Attention Visualization explainer for Transformer models.

Implements: ILocalExplainer<T, AttentionExplanation<T>>

For Beginners

Transformers use “attention” to decide which parts of the input to focus on when making predictions. This explainer visualizes these attention patterns.

What is attention?

  • Each position in the input “attends” to all other positions
  • Attention weights show how much each position influences another
  • Higher weight = more influence/importance

Types of attention in Transformers:

  1. Self-attention: Input attends to itself (e.g., words attending to other words)
  2. Cross-attention: One sequence attends to another (e.g., decoder attending to encoder)

Visualization methods:

  • Raw attention: Direct attention weights from the model
  • Attention rollout: Combines attention across all layers
  • Attention flow: Tracks information flow through the network

Example use cases:

  • NLP: See which words the model focuses on for classification
  • Vision Transformers: See which image patches are important
  • Time Series: See which past timesteps influence predictions

Multi-head attention:

  • Transformers have multiple “heads” that attend to different aspects
  • This explainer can show individual heads or their combination

Constructors

Constructor Summary
AttentionVisualizationExplainer(Func<Tensor<T>, Tensor<T>> predictFunction, Func<Tensor<T>, int, Tensor<T>> getAttentionWeights, int numLayers, int numHeads, int sequenceLength, string[] tokenLabels) Initializes a new Attention Visualization explainer.

Properties

Property Summary
bool SupportsGlobalExplanations { get; }
bool SupportsLocalExplanations { get; }
string MethodName { get; }

Methods

Method Summary
AttentionExplanation<T> Explain(Vector<T> instance) Visualizes attention patterns for an input.
AttentionExplanation<T> ExplainTensor(Tensor<T> input) Visualizes attention patterns for an input tensor.
AttentionExplanation<T>[] ExplainBatch(Matrix<T> instances)
Dictionary<int, T> GetAttentionFromPosition(AttentionExplanation<T> explanation, int queryPosition, int layer, int head) Gets attention for a specific token/position.
List<ValueTuple<int, string, T>> GetTopAttendedPositions(AttentionExplanation<T> explanation, int queryPosition, int topK, int layer) Gets the most attended positions for a query position.