StatisticalTestEngine<T>

Models & Types · AiDotNet.Evaluation.Engines

Engine for performing statistical tests on model comparison results.

For Beginners

When comparing machine learning models, you need statistical tests to determine if one model is truly better or if the difference is just due to random chance. This engine provides methods for:

  • Comparing two models (paired t-test, Wilcoxon, McNemar)
  • Comparing multiple models (Friedman test)
  • Post-hoc analysis (Nemenyi test)

Constructors

Constructor Summary
StatisticalTestEngine() Initializes the statistical test engine with all available tests.

Methods

Method Summary
ModelComparisonReport<T> CompareModels(T[][] scores, string[] modelNames, double alpha) Performs full comparison of multiple models with automatic test selection.
NemenyiResult<T> PostHocNemenyi(T[][] scores, double alpha) Performs Nemenyi post-hoc test after significant Friedman test.
StatisticalTestResult<T> CompareMultipleModelsFriedman(T[][] scores, double alpha) Compares multiple models using the Friedman test.
StatisticalTestResult<T> CompareTwoClassifiersMcNemar(T[] correctModel1, T[] correctModel2, double alpha) Compares two classifiers using McNemar’s test on their predictions.
StatisticalTestResult<T> CompareTwoModelsPairedTTest(T[] scoresModel1, T[] scoresModel2, double alpha) Compares two models using paired t-test (parametric).
StatisticalTestResult<T> CompareTwoModelsWilcoxon(T[] scoresModel1, T[] scoresModel2, double alpha) Compares two models using Wilcoxon signed-rank test (non-parametric).