IKnowledgeDistillationTrainer<T, TInput, TOutput>
Interfaces · AiDotNet.Interfaces
Defines the contract for knowledge distillation trainers that train student models using knowledge transferred from teacher models.
Known implementations: KnowledgeDistillationTrainerBase<T, TInput, TOutput>
For Beginners
A knowledge distillation trainer orchestrates the process of transferring knowledge from a large, accurate teacher model to a smaller, faster student model.
How It Works
Why an interface?
- Flexibility: Multiple trainer implementations (standard, self-distillation, multi-teacher, etc.)
- Testability: Easy to mock for unit testing
- Extensibility: New training strategies can be added without breaking existing code
- Dependency Injection: Can be injected into other components
Common Implementations:
- Standard Trainer: Single teacher → single student
- Self-Distillation Trainer: Model teaches itself (improves calibration)
- Multi-Teacher Trainer: Multiple teachers → one student (ensemble distillation)
- Online Trainer: Teacher updates during student training
- Mutual Learning Trainer: Multiple students learn from each other
Properties
| Property | Summary |
|---|---|
IDistillationStrategy<T> DistillationStrategy { get; } |
Gets the distillation strategy used for computing loss and gradients. |
ITeacherModel<TInput, TOutput> Teacher { get; } |
Gets the teacher model used for distillation. |
Methods
| Method | Summary |
|---|---|
T TrainBatch(Func<TInput, TOutput> studentForward, Action<TOutput> studentBackward, Vector<TInput> inputs, Vector<TOutput> trueLabels) |
Trains the student model on a single batch of data. |
double Evaluate(Func<TInput, TOutput> studentForward, Vector<TInput> testInputs, Vector<TOutput> testLabels) |
Evaluates the student model’s accuracy on test data. |
void Train(Func<TInput, TOutput> studentForward, Action<TOutput> studentBackward, Vector<TInput> trainInputs, Vector<TOutput> trainLabels, int epochs, int batchSize, Action<int, T> onEpochComplete) |
Trains the student model for multiple epochs. |