AUCPRMetric<T> |
Computes Area Under the Precision-Recall Curve (AUC-PR). |
AUCROCMetric<T> |
Computes Area Under the ROC Curve (AUC-ROC): measures discrimination ability. |
AccuracyMetric<T> |
Computes classification accuracy: the proportion of correct predictions. |
AdjustedR2Metric<T> |
Computes Adjusted R² Score: R² adjusted for the number of predictors in the model. |
BalancedAccuracyMetric<T> |
Computes balanced accuracy: the average recall across all classes. |
BalancedErrorRateMetric<T> |
Computes Balanced Error Rate (BER): average of FNR and FPR. |
BlockedKFoldStrategy<T> |
Blocked K-Fold: K-Fold with temporal blocking (gap) between train and validation. |
BootstrapStrategy<T> |
Bootstrap Cross-Validation: uses bootstrap sampling (sampling with replacement) for validation. |
BootstrapTest<T> |
Bootstrap-based hypothesis test for comparing two models. |
BrierScoreMetric<T> |
Computes Brier Score: mean squared error of probability predictions. |
CRPSMetric<T> |
Continuous Ranked Probability Score metric for probabilistic predictions. |
CalibrationBin<T> |
A single bin in the calibration curve. |
CalibrationEngine<T> |
Engine for analyzing and improving probability calibration of classifiers. |
CalibrationResult<T> |
Results from calibration analysis. |
ClassificationResults<T> |
Classification-specific results container. |
CochranQTest<T> |
Cochran’s Q test for comparing multiple classifiers on the same dataset. |
CohensKappaMetric<T> |
Computes Cohen’s Kappa: agreement measure that accounts for chance agreement. |
CrossValidationEngine<T> |
Engine for executing cross-validation with various strategies and aggregating results. |
CrossValidationResult<T> |
Aggregated results from cross-validation across all folds. |
DatasetStatistics<T> |
Statistics about the dataset used for evaluation. |
DeLongTest<T> |
DeLong’s test for comparing two ROC curves. |
DiagnosticOddsRatioMetric<T> |
Computes Diagnostic Odds Ratio: (TP × TN) / (FP × FN). |
ErrorRateMetric<T> |
Computes Error Rate (Misclassification Rate): 1 - Accuracy. |
EvaluationReport<T> |
Comprehensive evaluation report containing all computed metrics and analysis results. |
EvaluationWarning |
Warning generated during evaluation. |
ExplainedVarianceMetric<T> |
Computes Explained Variance Score: proportion of variance in target explained by predictions. |
F1ScoreMetric<T> |
Computes F1 score: the harmonic mean of precision and recall. |
FBetaScoreMetric<T> |
Computes F-beta score: weighted harmonic mean of precision and recall. |
FairnessEngine<T> |
Engine for analyzing fairness and bias in machine learning models. |
FairnessResult<T> |
Results from fairness analysis. |
FalseDiscoveryRateMetric<T> |
Computes False Discovery Rate (FDR): FP / (FP + TP) = 1 - Precision. |
FalseNegativeRateMetric<T> |
Computes False Negative Rate (FNR): proportion of actual positives incorrectly predicted as negative. |
FalseOmissionRateMetric<T> |
Computes False Omission Rate (FOR): FN / (FN + TN) = 1 - NPV. |
FalsePositiveRateMetric<T> |
Computes False Positive Rate (FPR): proportion of actual negatives incorrectly predicted as positive. |
FeatureStatistics<T> |
Statistics for a single feature. |
FoldResult<T> |
Results from a single cross-validation fold. |
FowlkesMallowsMetric<T> |
Computes the Fowlkes-Mallows Index: geometric mean of precision and recall. |
FriedmanTest<T> |
Friedman test: non-parametric test for comparing multiple classifiers across multiple datasets. |
GiniCoefficientMetric<T> |
Computes Gini Coefficient (normalized): 2 * AUC - 1, measures discriminative ability. |
GroupFairnessMetrics |
Fairness metrics for a specific group. |
GroupKFoldStrategy<T> |
Group K-Fold: K-Fold that keeps related samples (groups) together in the same fold. |
HammingLossMetric<T> |
Computes Hamming Loss: fraction of labels that are incorrectly predicted. |
HingeLossMetric<T> |
Computes Hinge Loss: loss function used by Support Vector Machines (SVM). |
HuberLossMetric<T> |
Computes Huber Loss: a robust loss function that is less sensitive to outliers than MSE. |
InformednessMetric<T> |
Computes Informedness (Youden’s J statistic): TPR + TNR - 1 = Recall + Specificity - 1. |
JaccardScoreMetric<T> |
Computes Jaccard Score (Intersection over Union): measures similarity between prediction and actual sets. |
KFoldStrategy<T> |
K-Fold cross-validation: splits data into K equal-sized folds, using each as validation once. |
KruskalWallisTest<T> |
Kruskal-Wallis H test for comparing multiple independent groups. |
LearningCurveEngine<T> |
Engine for generating learning curves: how model performance changes with training set size. |
LearningCurveResult<T> |
Results from learning curve analysis. |
LeaveOneOutStrategy<T> |
Leave-One-Out Cross-Validation (LOOCV): each sample is used once as validation. |
LeavePOutStrategy<T> |
Leave-P-Out cross-validation: train on N-P samples, validate on P samples for all combinations. |
LeveneTest<T> |
Levene’s test for equality of variances across groups. |
LogCoshLossMetric<T> |
Computes Log-Cosh Loss: mean of log(cosh(y - ŷ)). |
LogLossMetric<T> |
Computes Log Loss (Cross-Entropy Loss): a probabilistic measure that penalizes confident wrong predictions. |
LogScoreMetric<T> |
Logarithmic scoring metric for probabilistic predictions (Negative Log Likelihood). |
MAEMetric<T> |
Computes Mean Absolute Error (MAE): average absolute difference between predictions and actuals. |
MAPEMetric<T> |
Computes Mean Absolute Percentage Error (MAPE): average percentage error. |
MASEMetric<T> |
Computes Mean Absolute Scaled Error (MASE): scale-independent measure for time series. |
MASESeasonalMetric<T> |
Computes Seasonal MASE: Mean Absolute Scaled Error with explicit seasonal comparison. |
MSEMetric<T> |
Computes Mean Squared Error (MSE): average squared difference between predictions and actuals. |
MarkednessMetric<T> |
Computes Markedness: PPV + NPV - 1 = Precision + NPV - 1. |
MatthewsCorrelationCoefficientMetric<T> |
Computes Matthews Correlation Coefficient (MCC): a balanced measure for binary and multi-class classification. |
MaxErrorMetric<T> |
Computes Maximum Error: the worst-case absolute prediction error. |
McNemarTest<T> |
McNemar’s test: compares the performance of two binary classifiers on the same dataset. |
MeanAbsolutePercentageErrorMetric<T> |
Computes Mean Absolute Percentage Error with configurable handling of zeros. |
MeanBiasErrorMetric<T> |
Computes Mean Bias Error (MBE): average signed error showing systematic over/under-prediction. |
MeanDirectionalAccuracyMetric<T> |
Computes Mean Directional Accuracy (MDA): fraction of correctly predicted directions. |
MeanSquaredLogErrorMetric<T> |
Computes Mean Squared Logarithmic Error (MSLE): squared version of RMSLE. |
MedianAbsoluteErrorMetric<T> |
Computes Median Absolute Error (MedAE): the median of all absolute errors. |
MetricCollection<T> |
A collection of metrics, providing easy access by name and category. |
MetricComputationEngine<T> |
Core engine for computing evaluation metrics across all task types. |
MetricWithCI<T> |
Represents a metric value with optional confidence interval and metadata. |
ModelComparisonReport<T> |
Comprehensive report from model comparison. |
MonteCarloStrategy<T> |
Monte Carlo cross-validation (repeated random sub-sampling validation). |
NPVMetric<T> |
Computes Negative Predictive Value (NPV): proportion of negative predictions that are correct. |
NegativeLikelihoodRatioMetric<T> |
Computes Negative Likelihood Ratio (LR-): (1 - Sensitivity) / Specificity = FNR / TNR. |
NemenyiPostHocTest<T> |
Nemenyi post-hoc test: pairwise comparisons after Friedman test. |
NemenyiResult<T> |
Results from Nemenyi post-hoc test. |
NestedCVStrategy<T> |
Nested Cross-Validation: uses an inner CV loop for hyperparameter tuning and outer loop for evaluation. |
NormalizedMSEMetric<T> |
Computes Normalized Mean Squared Error (NMSE): MSE divided by variance of actuals. |
OptimizedPrecisionMetric<T> |
Computes Optimized Precision: Accuracy - |Sensitivity - Specificity| / (Sensitivity + Specificity). |
PairedTTest<T> |
Paired t-test: compares means of two related samples. |
PearsonCorrelationMetric<T> |
Computes Pearson Correlation Coefficient between predictions and actuals. |
PoissonDevianceMetric<T> |
Computes Poisson Deviance for count data regression. |
PositiveLikelihoodRatioMetric<T> |
Computes Positive Likelihood Ratio (LR+): Sensitivity / (1 - Specificity) = TPR / FPR. |
PositivePredictiveValueMetric<T> |
Computes Positive Predictive Value (PPV): same as Precision but named differently in medical contexts. |
PrecisionMetric<T> |
Computes precision (positive predictive value): the proportion of positive predictions that are correct. |
PrevalenceMetric<T> |
Computes Prevalence: fraction of actual positives in the dataset. |
PrevalenceThresholdMetric<T> |
Computes Prevalence Threshold: the prevalence at which the test would have 50% PPV. |
ProbabilityCalibrator<T> |
Calibrates model outputs to produce reliable probability estimates. |
PurgedKFoldStrategy<T> |
Purged K-Fold: K-Fold with temporal purging to prevent data leakage in financial/time-dependent data. |
QuantileLossMetric<T> |
Computes Quantile Loss (Pinball Loss) for quantile regression. |
R2ScoreMetric<T> |
Computes R² (coefficient of determination): proportion of variance explained by the model. |
RMSEMetric<T> |
Computes Root Mean Squared Error (RMSE): square root of MSE. |
RMSLEMetric<T> |
Computes Root Mean Squared Logarithmic Error (RMSLE): measures ratio errors rather than absolute errors. |
RecallMetric<T> |
Computes recall (sensitivity, true positive rate): the proportion of actual positives correctly identified. |
RegressionResults<T> |
Regression-specific results container. |
RelativeAbsoluteErrorMetric<T> |
Computes Relative Absolute Error (RAE): sum of absolute errors relative to baseline. |
RelativeSquaredErrorMetric<T> |
Computes Relative Squared Error (RSE): sum of squared errors relative to baseline model. |
RepeatedKFoldStrategy<T> |
Repeated K-Fold: runs K-Fold multiple times with different random shuffles. |
ResidualStatistics<T> |
Statistics about regression residuals. |
RobustnessEngine<T> |
Engine for analyzing model robustness to input perturbations and noise. |
RobustnessResult<T> |
Results from robustness analysis. |
SMAPEMetric<T> |
Computes Symmetric Mean Absolute Percentage Error (SMAPE): a bounded percentage error metric. |
ShuffleSplitStrategy<T> |
Shuffle Split (Monte Carlo Cross-Validation): random train/test splits repeated multiple times. |
SlidingWindowStrategy<T> |
Sliding Window cross-validation for time series with fixed-size training window. |
SpearmanCorrelationMetric<T> |
Computes Spearman’s Rank Correlation Coefficient between predictions and actuals. |
SpecificityMetric<T> |
Computes specificity (true negative rate): the proportion of actual negatives correctly identified. |
StatisticalTestEngine<T> |
Engine for performing statistical tests on model comparison results. |
StatisticalTestResult<T> |
Represents the result of a statistical test. |
StratifiedGroupKFoldStrategy<T> |
Stratified Group K-Fold: combines stratification and group constraints. |
StratifiedKFoldStrategy<T> |
Stratified K-Fold: K-Fold that preserves the percentage of samples for each class. |
SymmetricMAPEMetric<T> |
Computes Symmetric Mean Absolute Percentage Error (sMAPE) for regression. |
TheilUMetric<T> |
Computes Theil’s U Statistic: measures forecast accuracy relative to a naive no-change forecast. |
ThreatScoreMetric<T> |
Computes Threat Score (Critical Success Index): TP / (TP + FN + FP). |
TimeSeriesSplitStrategy<T> |
Time Series Split: expanding window cross-validation that respects temporal order. |
TrueNegativeRateMetric<T> |
Computes True Negative Rate (TNR): same as Specificity, TN / (TN + FP). |
TweedieLossMetric<T> |
Computes Tweedie Deviance Loss for regression with power parameter. |
ValidationCurveEngine<T> |
Engine for generating validation curves: how model performance changes with hyperparameter values. |
ValidationCurveResult<T> |
Results from validation curve analysis. |
WAPEMetric<T> |
Computes Weighted Absolute Percentage Error (WAPE): total absolute error as percentage of total actuals. |
WeightedMAPEMetric<T> |
Computes Weighted Mean Absolute Percentage Error (wMAPE): weighted by actual values. |
WilcoxonSignedRankTest<T> |
Wilcoxon signed-rank test: non-parametric paired comparison test. |
ZeroOneLossMetric<T> |
Computes Zero-One Loss: fraction of misclassifications (complement of accuracy). |