BackgroundSummarizer<T>
Helpers & Utilities · AiDotNet.Interpretability.Helpers
Provides methods for summarizing background data for SHAP and other interpretability methods.
For Beginners
SHAP and similar methods need “background data” to compare against. But using all your training data as background can be VERY slow.
This class provides ways to summarize large datasets into smaller representative sets:
- KMeans: Group similar samples together, use cluster centers as representatives
- Stratified Sampling: Sample proportionally from different groups/categories
- Random Sampling: Simple random subset selection
Example: If you have 10,000 training samples, you might summarize to 100 samples. SHAP computation goes from O(10000) to O(100) = 100x faster!
Trade-off:
- Fewer background samples = faster but less accurate
- More background samples = slower but more accurate
- 50-200 samples is usually a good balance
Methods
| Method | Summary |
|---|---|
BackgroundSummary<T> Auto(Matrix<T> data, int nSamples, int[] categoricalColumns, int? randomState) |
Automatically chooses the best summarization method. |
BackgroundSummary<T> KMeans(Matrix<T> data, int k, int maxIterations, int? randomState) |
Summarizes data using KMeans clustering. |
BackgroundSummary<T> MixedSummary(Matrix<T> data, int[] categoricalColumns, int nSamples, int? randomState) |
Summarizes data using a combination of methods for mixed data. |
BackgroundSummary<T> RandomSample(Matrix<T> data, int nSamples, int? randomState) |
Summarizes data using random sampling. |
BackgroundSummary<T> StratifiedSample(Matrix<T> data, int stratifyColumn, int nPerStratum, int? randomState) |
Summarizes data using stratified sampling. |