IntermediateActivations<T>
Models & Types · AiDotNet.KnowledgeDistillation
Stores intermediate layer activations collected during a forward pass.
For Beginners
During neural network training, we often want to inspect or use the outputs from internal layers (not just the final output). These internal outputs are called “intermediate activations”. This class stores them in a dictionary keyed by layer name.
How It Works
Use Cases:
- Feature-based distillation: Match intermediate layer outputs between teacher and student
- Neuron selectivity: Analyze how individual neurons respond across a batch
- Attention transfer: Transfer attention patterns from teacher to student
- Debugging: Inspect what each layer is learning
Example Usage:
Constructors
| Constructor | Summary |
|---|---|
IntermediateActivations() |
Properties
| Property | Summary |
|---|---|
IEnumerable<string> LayerNames { get; } |
Gets the names of all layers with stored activations. |
IReadOnlyDictionary<string, Matrix<T>> AllActivations { get; } |
Gets all stored activations as a read-only dictionary. |
int LayerCount { get; } |
Gets the number of layers with stored activations. |
Methods
| Method | Summary |
|---|---|
Matrix<T> Get(string layerName) |
Retrieves intermediate activations for a specific layer. |
bool Contains(string layerName) |
Checks if activations exist for a specific layer. |
void Add(string layerName, Matrix<T> activations) |
Adds intermediate activations for a specific layer. |