AdaptiveDistillationStrategyBase<T>

Base Classes ยท AiDotNet.KnowledgeDistillation.Strategies

Abstract base class for adaptive distillation strategies with performance tracking.

Inheritance: DistillationStrategyBase<T> โ†’ AdaptiveDistillationStrategyBase<T>

Implements: IAdaptiveDistillationStrategy<T>

Known derived types: AccuracyBasedAdaptiveStrategy<T>, ConfidenceBasedAdaptiveStrategy<T>, EntropyBasedAdaptiveStrategy<T>

For Beginners

This base class provides common functionality for all adaptive strategies, including performance tracking with exponential moving average and temperature range management.

How It Works

For Implementers: Derive from this class and implement ComputeAdaptiveTemperature to define your specific adaptation logic.

Shared Features:

  • Exponential moving average (EMA) for performance tracking
  • Temperature range validation and enforcement
  • Performance history management
  • Helper methods for confidence, entropy, and accuracy calculations

Properties

Property Summary
double AdaptationRate { get; } Gets the adaptation rate for exponential moving average.
double MaxTemperature { get; } Gets the maximum temperature for adaptation.
double MinTemperature { get; } Gets the minimum temperature for adaptation.

Methods

Method Summary
Matrix<T> ComputeGradient(Matrix<T> studentBatchOutput, Matrix<T> teacherBatchOutput, Matrix<T> trueLabelsBatch) Computes gradient with adaptive temperature.
T ComputeLoss(Matrix<T> studentBatchOutput, Matrix<T> teacherBatchOutput, Matrix<T> trueLabelsBatch) Computes distillation loss with adaptive temperature.
double ComputeAdaptiveTemperature(Vector<T> studentOutput, Vector<T> teacherOutput) Computes the adaptive temperature for a specific sample.
double GetPerformance(int sampleIndex) Gets the current performance metric for a sample.
void UpdatePerformance(int sampleIndex, Vector<T> studentOutput, Vector<T> trueLabel) Updates the performance metric for a specific sample using exponential moving average.