EasyToHardCurriculumStrategy<T>
Models & Types · AiDotNet.KnowledgeDistillation.Strategies
Curriculum distillation strategy that progresses from easy to hard samples.
Inheritance: DistillationStrategyBase<T> → CurriculumDistillationStrategyBase<T> → EasyToHardCurriculumStrategy<T>
For Beginners
Easy-to-hard curriculum learning mimics how humans learn best: start with simple concepts and gradually introduce more complex ones. This strategy filters training samples and adjusts temperature based on difficulty and training progress.
How It Works
How It Works:
- Early Training (progress 0.0-0.3):
- Include only easy samples (difficulty ≤ 0.3)
- Use high temperature (soft targets, gentle learning)
- Mid Training (progress 0.3-0.7):
- Include easy and medium samples (difficulty ≤ 0.7)
- Gradually decrease temperature
- Late Training (progress 0.7-1.0):
- Include all samples (even hard ones)
- Use low temperature (sharp targets, challenging)
Temperature Progression: Starts at MaxTemperature (e.g., 5.0) and linearly decreases to MinTemperature (e.g., 2.0) as training progresses. This makes distillation progressively more challenging.
Sample Filtering: At progress P, only include samples with difficulty ≤ P. Example: At 50% progress, only samples with difficulty ≤ 0.5 are included.
Real-World Analogy: Learning mathematics: Start with addition (easy), then multiplication (medium), then algebra (hard). Don’t try to teach calculus to someone who hasn’t learned addition!
Best For:
- Training from scratch
- Datasets with clear difficulty levels
- Preventing student from being overwhelmed early
- Improving convergence speed and final performance
References:
- Bengio et al. (2009). Curriculum Learning. ICML.
- Kumar et al. (2010). Self-paced Learning for Latent Variable Models.
Constructors
| Constructor | Summary |
|---|---|
EasyToHardCurriculumStrategy(double baseTemperature, double alpha, double minTemperature, double maxTemperature, int totalSteps, Dictionary<int, double> sampleDifficulties) |
Initializes a new instance of the EasyToHardCurriculumStrategy class. |
Methods
| Method | Summary |
|---|---|
bool ShouldIncludeSample(int sampleIndex) |
Determines if a sample should be included based on curriculum progress. |
double ComputeCurriculumTemperature() |
Computes curriculum temperature that decreases over time (easy to hard). |