ProbabilityCalibrator<T>
Models & Types · AiDotNet.Evaluation.Calibration
Calibrates model outputs to produce reliable probability estimates.
For Beginners
Calibration is like adjusting a thermometer. Your model might always say “90 degrees” when the true temperature is 85. Calibration learns this systematic error and corrects for it.
Available methods:
- Platt Scaling: Fits a logistic curve - good for SVMs and many classifiers
- Isotonic Regression: Non-parametric curve fitting - more flexible but needs more data
- Temperature Scaling: Simple division - popular for neural networks
- Histogram Binning: Averages within bins - simple and interpretable
- Beta Calibration: Fits beta distribution - good for bounded outputs
Example usage:
- Train your model and get predicted probabilities
- Fit the calibrator on a held-out validation set
- Use calibrator to transform all future predictions
How It Works
Probability calibration transforms raw model scores into well-calibrated probabilities. A well-calibrated model means: if you see 1000 predictions of “70% probability”, about 700 of those should be positive outcomes.
Constructors
| Constructor | Summary |
|---|---|
ProbabilityCalibrator(ProbabilityCalibratorOptions options) |
Initializes a new instance of the calibrator. |
Properties
| Property | Summary |
|---|---|
ProbabilityCalibrationMethod ProbabilityCalibrationMethod { get; } |
Gets the calibration method being used. |
Methods
| Method | Summary |
|---|---|
T GetTemperature() |
Gets the temperature parameter. |
ValueTuple<T, T, T> GetBetaParameters() |
Gets the beta calibration parameters (a, b, c). |
ValueTuple<T, T> GetPlattParameters() |
Gets the Platt scaling parameters (A, B). |
ValueTuple<T[], T[], int[]> GetReliabilityDiagram(Vector<T> scores, Vector<T> labels, int numBins) |
Returns mean predictions, positive fractions and exact counts per uniform bin. |
Vector<T> FitTransform(Vector<T> scores, Vector<T> labels) |
Fits the calibrator and transforms in one step. |
Vector<T> Transform(Vector<T> scores) |
Transforms predicted scores into calibrated probabilities. |
double ComputeBrierScore(Vector<T> scores, Vector<T> labels) |
Computes the mean squared difference between scores and labels. |
double ComputeECE(Vector<T> scores, Vector<T> labels, int numBins) |
Computes the Expected Calibration Error (ECE). |
double ComputeMCE(Vector<T> scores, Vector<T> labels, int numBins) |
Computes the largest absolute calibration error among non-empty bins. |
void Fit(Vector<T> scores, Vector<T> labels) |
Fits the calibrator to predicted scores and true labels. |