IntermediateActivations<T>

Models & Types · AiDotNet.KnowledgeDistillation

Stores intermediate layer activations collected during a forward pass.

For Beginners

During neural network training, we often want to inspect or use the outputs from internal layers (not just the final output). These internal outputs are called “intermediate activations”. This class stores them in a dictionary keyed by layer name.

How It Works

Use Cases:

  • Feature-based distillation: Match intermediate layer outputs between teacher and student
  • Neuron selectivity: Analyze how individual neurons respond across a batch
  • Attention transfer: Transfer attention patterns from teacher to student
  • Debugging: Inspect what each layer is learning

Example Usage:

Constructors

Constructor Summary
IntermediateActivations()

Properties

Property Summary
IEnumerable<string> LayerNames { get; } Gets the names of all layers with stored activations.
IReadOnlyDictionary<string, Matrix<T>> AllActivations { get; } Gets all stored activations as a read-only dictionary.
int LayerCount { get; } Gets the number of layers with stored activations.

Methods

Method Summary
Matrix<T> Get(string layerName) Retrieves intermediate activations for a specific layer.
bool Contains(string layerName) Checks if activations exist for a specific layer.
void Add(string layerName, Matrix<T> activations) Adds intermediate activations for a specific layer.