AdaptiveDistillationStrategyBase<T>
Base Classes ยท AiDotNet.KnowledgeDistillation.Strategies
Abstract base class for adaptive distillation strategies with performance tracking.
Inheritance: DistillationStrategyBase<T> โ AdaptiveDistillationStrategyBase<T>
Implements: IAdaptiveDistillationStrategy<T>
Known derived types: AccuracyBasedAdaptiveStrategy<T>, ConfidenceBasedAdaptiveStrategy<T>, EntropyBasedAdaptiveStrategy<T>
For Beginners
This base class provides common functionality for all adaptive strategies, including performance tracking with exponential moving average and temperature range management.
How It Works
For Implementers: Derive from this class and implement
ComputeAdaptiveTemperature to define your specific adaptation logic.
Shared Features:
- Exponential moving average (EMA) for performance tracking
- Temperature range validation and enforcement
- Performance history management
- Helper methods for confidence, entropy, and accuracy calculations
Properties
| Property | Summary |
|---|---|
double AdaptationRate { get; } |
Gets the adaptation rate for exponential moving average. |
double MaxTemperature { get; } |
Gets the maximum temperature for adaptation. |
double MinTemperature { get; } |
Gets the minimum temperature for adaptation. |
Methods
| Method | Summary |
|---|---|
Matrix<T> ComputeGradient(Matrix<T> studentBatchOutput, Matrix<T> teacherBatchOutput, Matrix<T> trueLabelsBatch) |
Computes gradient with adaptive temperature. |
T ComputeLoss(Matrix<T> studentBatchOutput, Matrix<T> teacherBatchOutput, Matrix<T> trueLabelsBatch) |
Computes distillation loss with adaptive temperature. |
double ComputeAdaptiveTemperature(Vector<T> studentOutput, Vector<T> teacherOutput) |
Computes the adaptive temperature for a specific sample. |
double GetPerformance(int sampleIndex) |
Gets the current performance metric for a sample. |
void UpdatePerformance(int sampleIndex, Vector<T> studentOutput, Vector<T> trueLabel) |
Updates the performance metric for a specific sample using exponential moving average. |