NeuronAttributionExplainer<T>
Models & Types · AiDotNet.Interpretability.Explainers
Neuron-level attribution explainer for understanding individual neuron contributions.
Implements: IGPUAcceleratedExplainer<T>
For Beginners
While most attribution methods explain which INPUT FEATURES matter, neuron attribution explains which NEURONS IN A LAYER contribute to the output.
Why is this useful?
- Understanding hidden representations: What did the model learn in each layer?
- Feature discovery: Which neurons encode which concepts?
- Debugging: Are certain neurons always/never active?
- Pruning: Which neurons can be removed without hurting performance?
Supported methods:
- NeuronGradient: Simple gradient of output w.r.t. neuron activation
- NeuronIntegratedGradients: Integrated Gradients from baseline to actual activation
- NeuronConductance: Combines gradient and activation (like Input×Gradient for neurons)
Example use case: In a CNN for image classification, you might find that neuron #42 in the last conv layer has high attribution for “cat” predictions. Investigating what activates neuron #42 could reveal it’s a “whisker detector”.
Constructors
| Constructor | Summary |
|---|---|
NeuronAttributionExplainer(Func<Vector<T>, Vector<T>> predictFunction, Func<Vector<T>, Vector<T>> layerActivationFunction, Func<Vector<T>, int, int, T> neuronGradientFunction, int layerSize, NeuronAttributionMethod method, int integrationSteps, string[] neuronNames) |
Initializes a neuron attribution explainer. |
Properties
| Property | Summary |
|---|---|
bool IsGPUAccelerated { get; } |
|
bool SupportsGlobalExplanations { get; } |
Gets whether this explainer supports global explanations. |
bool SupportsLocalExplanations { get; } |
Gets whether this explainer supports local explanations. |
string MethodName { get; } |
Gets the method name. |
Methods
| Method | Summary |
|---|---|
NeuronAttributionResult<T> ComputeNeuronAttribution(Vector<T> instance, int? targetClass) |
Computes neuron attribution for all neurons in the layer. |
void SetGPUHelper(GPUExplainerHelper<T> helper) |