ConfidenceBasedAdaptiveStrategy<T>

Models & Types · AiDotNet.KnowledgeDistillation.Strategies

Adaptive distillation strategy that adjusts temperature based on student confidence.

Inheritance: DistillationStrategyBase<T> → AdaptiveDistillationStrategyBase<T> → ConfidenceBasedAdaptiveStrategy<T>

For Beginners

This strategy adapts temperature based on how confident the student is in its predictions. When the student is confident (high max probability), we use lower temperature (harder distillation). When uncertain (low max probability), we use higher temperature (easier distillation with softer targets).

How It Works

Intuition:

  • High Confidence → Student understands this sample → Lower temp (sharpen targets)
  • Low Confidence → Student struggles with this sample → Higher temp (soften targets)

Example: Student predicts [0.95, 0.03, 0.02] → High confidence (0.95) → Low temperature Student predicts [0.40, 0.35, 0.25] → Low confidence (0.40) → High temperature

Best For:

  • General-purpose adaptive distillation
  • When you want automatic difficulty adjustment
  • Datasets with varying sample complexity

Temperature Mapping: Confidence = max(probabilities) Difficulty = 1 - Confidence Temperature = MinTemp + Difficulty * (MaxTemp - MinTemp)

Constructors

Constructor Summary
ConfidenceBasedAdaptiveStrategy(double baseTemperature, double alpha, double minTemperature, double maxTemperature, double adaptationRate) Initializes a new instance of the ConfidenceBasedAdaptiveStrategy class.

Methods

Method Summary
double ComputeAdaptiveTemperature(Vector<T> studentOutput, Vector<T> teacherOutput) Computes adaptive temperature based on student confidence.