TeacherModelType

Enums · AiDotNet.Enums

Specifies the type of teacher model to use for knowledge distillation.

For Beginners

The teacher model is the “expert” that guides the student model’s learning. Different teacher types are suited for different scenarios and distillation goals.

How It Works

Choosing a Teacher:

  • Use NeuralNetwork for standard NN-to-NN distillation
  • Use Ensemble to combine knowledge from multiple models
  • Use Pretrained to load from checkpoints or ONNX
  • Use Adaptive for curriculum learning (progressive difficulty)
  • Use Online when teacher should update during training

Fields

Field Summary
TeacherModelType Adaptive Adaptive teacher that adjusts teaching based on student performance.
TeacherModelType Curriculum Curriculum teacher that provides progressive difficulty.
TeacherModelType Distributed Distributed teacher split across multiple devices/nodes.
TeacherModelType Ensemble Ensemble of multiple teacher models.
TeacherModelType MultiModal Multi-modal teacher (e.g., CLIP, vision-language models).
TeacherModelType NeuralNetwork Standard neural network teacher.
TeacherModelType Online Online teacher that updates during student training.
TeacherModelType Pretrained Pretrained model loaded from checkpoint or ONNX.
TeacherModelType Quantized Quantized teacher with reduced precision (INT8, INT4).
TeacherModelType Self Self-teacher where model teaches itself (Born-Again Networks).
TeacherModelType Transformer Transformer-based teacher (BERT, GPT, ViT, etc.).
int value__