NeuronAttributionExplainer<T>

Models & Types · AiDotNet.Interpretability.Explainers

Neuron-level attribution explainer for understanding individual neuron contributions.

Implements: IGPUAcceleratedExplainer<T>

For Beginners

While most attribution methods explain which INPUT FEATURES matter, neuron attribution explains which NEURONS IN A LAYER contribute to the output.

Why is this useful?

  • Understanding hidden representations: What did the model learn in each layer?
  • Feature discovery: Which neurons encode which concepts?
  • Debugging: Are certain neurons always/never active?
  • Pruning: Which neurons can be removed without hurting performance?

Supported methods:

  • NeuronGradient: Simple gradient of output w.r.t. neuron activation
  • NeuronIntegratedGradients: Integrated Gradients from baseline to actual activation
  • NeuronConductance: Combines gradient and activation (like Input×Gradient for neurons)

Example use case: In a CNN for image classification, you might find that neuron #42 in the last conv layer has high attribution for “cat” predictions. Investigating what activates neuron #42 could reveal it’s a “whisker detector”.

Constructors

Constructor Summary
NeuronAttributionExplainer(Func<Vector<T>, Vector<T>> predictFunction, Func<Vector<T>, Vector<T>> layerActivationFunction, Func<Vector<T>, int, int, T> neuronGradientFunction, int layerSize, NeuronAttributionMethod method, int integrationSteps, string[] neuronNames) Initializes a neuron attribution explainer.

Properties

Property Summary
bool IsGPUAccelerated { get; }
bool SupportsGlobalExplanations { get; } Gets whether this explainer supports global explanations.
bool SupportsLocalExplanations { get; } Gets whether this explainer supports local explanations.
string MethodName { get; } Gets the method name.

Methods

Method Summary
NeuronAttributionResult<T> ComputeNeuronAttribution(Vector<T> instance, int? targetClass) Computes neuron attribution for all neurons in the layer.
void SetGPUHelper(GPUExplainerHelper<T> helper)