ConformalClassifier<T>

Models & Types · AiDotNet.UncertaintyQuantification.ConformalPrediction

Implements Conformal Prediction for classification tasks.

For Beginners

Conformal Prediction for classification provides prediction sets with guaranteed coverage, not just a single class prediction.

Key differences from regression conformal prediction:

  • Instead of an interval, you get a SET of possible classes
  • The set is guaranteed to contain the true class with specified probability

Example with 90% confidence:

  • Traditional classifier: “This is a cat” (might be wrong)
  • Conformal classifier: “This is {cat, dog}” (90% guaranteed to include correct class)

Benefits:

  • When the model is uncertain, you get a larger prediction set (e.g., {cat, dog, rabbit})
  • When the model is confident, you get a smaller set (e.g., {cat})
  • You can defer to a human expert when the prediction set is too large

This is invaluable for:

  • Medical diagnosis: Know when to seek specialist opinion
  • Autonomous systems: Know when to hand control back to operator
  • Quality control: Flag uncertain cases for manual review

Constructors

Constructor Summary
ConformalClassifier(INeuralNetwork<T> model, int numClasses) Initializes a new instance of the ConformalClassifier class.

Methods

Method Summary
HashSet<int> PredictSet(Tensor<T> input, double confidenceLevel) Predicts with a conformal prediction set.
T ComputeAverageSetSize(Matrix<T> testInputs, double confidenceLevel) Computes the average size of prediction sets on a test set.
T EvaluateCoverage(Matrix<T> testInputs, Vector<int> testLabels, double confidenceLevel) Evaluates the empirical coverage on a test set.
void Calibrate(Matrix<T> calibrationInputs, Vector<int> calibrationLabels) Calibrates the conformal classifier using a calibration dataset.