ModelComparisonOptions
Options & Configuration · AiDotNet.Evaluation.Options
Configuration options for comparing multiple models.
For Beginners
When comparing models, you might find Model A has 85% accuracy and Model B has 87% accuracy. But is that 2% difference real or just noise? Statistical tests help answer this by computing p-values (probability the difference is due to chance).
How It Works
Model comparison provides statistical tests to determine if one model is significantly better than another, or to rank multiple models.
Constructors
| Constructor | Summary |
|---|---|
ModelComparisonOptions() |
Properties
| Property | Summary |
|---|---|
CrossValidationStrategy? CVStrategy { get; set; } |
Cross-validation strategy for comparison. |
EffectSizeMeasure? EffectSizeMeasure { get; set; } |
Effect size measure to use. |
MultipleComparisonTest? MultipleComparisonTest { get; set; } |
Test for comparing multiple models simultaneously. |
MultipleTestingCorrectionMethod? CorrectionMethod { get; set; } |
Multiple testing correction method. |
PairwiseComparisonTest? PairwiseTest { get; set; } |
Statistical test for pairwise comparison. |
PostHocTest? PostHocTest { get; set; } |
Post-hoc test after significant Friedman test. |
bool? ApplyMultipleTestingCorrection { get; set; } |
Whether to apply multiple testing correction. |
bool? ComputeConfidenceIntervals { get; set; } |
Whether to compute confidence intervals for differences. |
bool? ComputeEffectSizes { get; set; } |
Whether to compute effect sizes. |
bool? GenerateCriticalDifferenceDiagram { get; set; } |
Whether to create critical difference diagram data. |
bool? GenerateRanking { get; set; } |
Whether to generate ranking of models. |
bool? ParallelExecution { get; set; } |
Whether to run comparisons in parallel. |
bool? UseBayesianComparison { get; set; } |
Whether to use Bayesian comparison. |
double? ConfidenceLevel { get; set; } |
Confidence level for intervals. |
double? RopeWidth { get; set; } |
ROPE (Region of Practical Equivalence) for Bayesian comparison. |
double? SignificanceLevel { get; set; } |
Significance level (alpha) for hypothesis tests. |
int? BootstrapSamples { get; set; } |
Number of bootstrap samples. |
int? CVFolds { get; set; } |
Number of CV folds. |
int? CVRepeats { get; set; } |
Number of CV repetitions. |
int? RandomSeed { get; set; } |
Random seed for reproducibility. |
string PrimaryMetric { get; set; } |
Primary metric for comparison. |