StatisticalTestEngine<T>
Models & Types · AiDotNet.Evaluation.Engines
Engine for performing statistical tests on model comparison results.
For Beginners
When comparing machine learning models, you need statistical tests to determine if one model is truly better or if the difference is just due to random chance. This engine provides methods for:
- Comparing two models (paired t-test, Wilcoxon, McNemar)
- Comparing multiple models (Friedman test)
- Post-hoc analysis (Nemenyi test)
Constructors
| Constructor | Summary |
|---|---|
StatisticalTestEngine() |
Initializes the statistical test engine with all available tests. |
Methods
| Method | Summary |
|---|---|
ModelComparisonReport<T> CompareModels(T[][] scores, string[] modelNames, double alpha) |
Performs full comparison of multiple models with automatic test selection. |
NemenyiResult<T> PostHocNemenyi(T[][] scores, double alpha) |
Performs Nemenyi post-hoc test after significant Friedman test. |
StatisticalTestResult<T> CompareMultipleModelsFriedman(T[][] scores, double alpha) |
Compares multiple models using the Friedman test. |
StatisticalTestResult<T> CompareTwoClassifiersMcNemar(T[] correctModel1, T[] correctModel2, double alpha) |
Compares two classifiers using McNemar’s test on their predictions. |
StatisticalTestResult<T> CompareTwoModelsPairedTTest(T[] scoresModel1, T[] scoresModel2, double alpha) |
Compares two models using paired t-test (parametric). |
StatisticalTestResult<T> CompareTwoModelsWilcoxon(T[] scoresModel1, T[] scoresModel2, double alpha) |
Compares two models using Wilcoxon signed-rank test (non-parametric). |