TCAVExplainer<T>
Models & Types · AiDotNet.Interpretability.Explainers
Testing with Concept Activation Vectors (TCAV) explainer.
Implements: IGPUAcceleratedExplainer<T>
For Beginners
TCAV is a technique that explains model predictions using human-understandable concepts instead of raw features. While most explanation methods tell you which pixels or features matter, TCAV tells you which CONCEPTS matter.
Example: Instead of highlighting pixels that matter for a “doctor” prediction, TCAV can tell you whether the concept of “stethoscope” or “white coat” influenced the prediction.
How TCAV works:
- Collect concept examples: Gather images/examples that represent your concept
(e.g., images with stripes for a “striped” concept)
- Collect random examples: Gather examples that don’t specifically represent the concept
- Train a CAV: Train a linear classifier to distinguish concept vs random at a
specific layer. The classifier’s weight vector becomes the Concept Activation Vector (CAV).
- Compute directional derivatives: See how sensitive the model output is to moving
in the direction of the CAV
- Compute TCAV score: The fraction of test inputs where moving toward the concept
increases the model’s prediction for a class
Interpreting TCAV scores:
- TCAV score = 0.8 means 80% of images are more likely to be classified as the target
class when they have more of the concept
- TCAV score = 0.5 means the concept doesn’t influence the prediction (random)
- TCAV score = 0.2 means the concept DECREASES the likelihood of the target class
Statistical significance: TCAV runs multiple times with different random samples to ensure the score is statistically significant and not due to random noise.
When to use TCAV:
- When you want high-level, human-understandable explanations
- When you have examples of concepts you want to test
- For testing fairness/bias (e.g., does “gender” concept affect hiring predictions?)
- For debugging models (e.g., is my model relying on spurious correlations?)
Constructors
| Constructor | Summary |
|---|---|
TCAVExplainer(Func<Vector<T>, Vector<T>> predictFunction, Func<Vector<T>, Vector<T>> layerActivationFunction, Func<Vector<T>, int, Vector<T>> gradientToLayerFunction, int layerSize, double regularization, int numCavs, double significanceLevel, int? randomState) |
Initializes a new TCAV explainer. |
Properties
| Property | Summary |
|---|---|
bool IsGPUAccelerated { get; } |
|
bool SupportsGlobalExplanations { get; } |
Gets whether this explainer supports global explanations. |
bool SupportsLocalExplanations { get; } |
Gets whether this explainer supports local explanations. |
string MethodName { get; } |
Gets the method name. |
Methods
| Method | Summary |
|---|---|
ConceptActivationVector<T> TrainCAV(Matrix<T> conceptExamples, Matrix<T> randomExamples) |
Trains a Concept Activation Vector from concept and random examples. |
LocalTCAVExplanation<T> ExplainLocal(Vector<T> input, ConceptActivationVector<T> cav, int targetClass, string conceptName) |
Computes concept sensitivity for a single input. |
T ComputeDirectionalDerivative(Vector<T> input, ConceptActivationVector<T> cav, int targetClass) |
Computes the directional derivative of the model output with respect to the CAV. |
TCAVExplainer<T> FromNetwork(INeuralNetwork<T> network, Func<Vector<T>, Vector<T>> layerActivationFunction, int layerSize, int numCavs, double significanceLevel, int? randomState) |
Creates a TCAV explainer from a neural network using input gradient helper. |
TCAVResult<T> RunTCAV(Matrix<T> conceptExamples, Matrix<T> randomExamplesPool, Matrix<T> testInputs, int targetClass, string conceptName) |
Runs a full TCAV analysis with statistical significance testing. |
TCAVResults<T> RunTCAVMultiple(Dictionary<string, Matrix<T>> concepts, Matrix<T> randomExamplesPool, Matrix<T> testInputs, int targetClass) |
Runs TCAV analysis for multiple concepts. |
double ComputeTCAVScore(Matrix<T> testInputs, ConceptActivationVector<T> cav, int targetClass) |
Computes the TCAV score for a concept on a set of inputs. |
void SetGPUHelper(GPUExplainerHelper<T> helper) |