TeacherModelWrapper<T>
Models & Types · AiDotNet.KnowledgeDistillation
Wraps an existing trained IFullModel to act as a teacher for knowledge distillation.
Implements: ITeacherModel<Vector<T>, Vector<T>>
For Beginners
This class takes any trained IFullModel and adapts it to work as a teacher in knowledge distillation. The teacher model should already be trained and perform well on your task.
How It Works
Architecture Note: This is a lightweight adapter that bridges IFullModel to ITeacherModel. It simply delegates GetLogits() to the underlying model’s Predict() method, since in this architecture, predictions and logits are equivalent.
Real-world Example: Imagine you have a large, accurate neural network trained on your dataset. You can wrap it with TeacherModelWrapper and use it to train a smaller, faster student model that retains most of the accuracy but runs much faster.
Common teacher-student scenarios:
- Large neural network (teacher) → Smaller network (student): 40-60% smaller, 95-97% of performance
- Deep network (teacher) → Shallow network (student): 10x faster inference
- Ensemble (teacher) → Single model (student): Deployable on resource-constrained devices
Constructors
| Constructor | Summary |
|---|---|
TeacherModelWrapper(Func<Vector<T>, Vector<T>> forwardFunc, int outputDimension) |
Initializes a new instance of the TeacherModelWrapper class from a forward function. |
Properties
| Property | Summary |
|---|---|
int OutputDimension { get; } |
Gets the number of output dimensions (e.g., number of classes for classification). |
Methods
| Method | Summary |
|---|---|
Vector<T> GetLogits(Vector<T> input) |
Gets the teacher’s raw logits (pre-softmax outputs) for the given input. |