KnowledgeDistillationOptions<T, TInput, TOutput>
Options & Configuration · AiDotNet.Models.Options
Configuration options for knowledge distillation training.
Inheritance: ModelOptions → KnowledgeDistillationOptions<T, TInput, TOutput>
For Beginners
This class configures how knowledge distillation works. Think of it as the “settings” for transferring knowledge from a large teacher model to a smaller student model.
How It Works
Quick Start Example:
Constructors
| Constructor | Summary |
|---|---|
KnowledgeDistillationOptions() |
Properties
| Property | Summary |
|---|---|
Action<int, T> OnEpochComplete { get; set; } |
Gets or sets callback function invoked after each epoch. |
Func<TInput, TOutput> TeacherForward { get; set; } |
Gets or sets the teacher model forward function (alternative approach). |
IDistillationStrategy<T> Strategy { get; set; } |
Gets or sets the distillation strategy. |
IFullModel<T, TInput, TOutput> TeacherModel { get; set; } |
Gets or sets the teacher IFullModel (recommended approach). |
ITeacherModel<TInput, TOutput> Teacher { get; set; } |
Gets or sets the teacher model instance (if using pre-instantiated teacher). |
TInput[] ValidationInputs { get; set; } |
Gets or sets validation data inputs (if validation is enabled). |
TOutput[] ValidationLabels { get; set; } |
Gets or sets validation data labels (if validation is enabled). |
TeacherModelType TeacherModelType { get; set; } |
Gets or sets the type of teacher model to use. |
Vector<ITeacherModel<TInput, TOutput>> Teachers { get; set; } |
Gets or sets multiple teacher models (for ensemble distillation). |
Vector<double> EnsembleWeights { get; set; } |
Gets or sets ensemble weights (if using multiple teachers). |
Vector<string> AttentionLayers { get; set; } |
Gets or sets attention layer names (if using attention-based distillation). |
Vector<string> FeatureLayerPairs { get; set; } |
Gets or sets layer pairs for feature-based distillation. |
bool FreezeTeacher { get; set; } |
Gets or sets whether to freeze teacher model during training. |
bool SaveCheckpoints { get; set; } |
Gets or sets whether to save checkpoints during training. |
bool SaveOnlyBestCheckpoint { get; set; } |
Gets or sets whether to only save the best model checkpoint. |
bool UseEMA { get; set; } |
Gets or sets whether to use exponential moving average for teacher predictions (self-distillation). |
bool UseEarlyStopping { get; set; } |
Gets or sets whether to enable early stopping based on validation loss. |
bool UseLabelSmoothing { get; set; } |
Gets or sets whether to use label smoothing. |
bool ValidateAfterEpoch { get; set; } |
Gets or sets whether to validate model after each epoch. |
double Alpha { get; set; } |
Gets or sets the alpha parameter balancing hard and soft loss. |
double AttentionWeight { get; set; } |
Gets or sets weight for attention loss (if using attention-based distillation). |
double EMADecay { get; set; } |
Gets or sets the EMA decay rate (if using EMA). |
double EarlyStoppingMinDelta { get; set; } |
Gets or sets minimum improvement delta for early stopping. |
double FeatureWeight { get; set; } |
Gets or sets weight for feature loss (if using feature-based distillation). |
double LabelSmoothingFactor { get; set; } |
Gets or sets the label smoothing factor (if enabled). |
double LearningRate { get; set; } |
Gets or sets the learning rate for student training. |
double Temperature { get; set; } |
Gets or sets the temperature for softmax scaling. |
int BatchSize { get; set; } |
Gets or sets the batch size for training. |
int CheckpointFrequency { get; set; } |
Gets or sets checkpoint frequency (save every N epochs). |
int EarlyStoppingPatience { get; set; } |
Gets or sets patience for early stopping (epochs without improvement). |
int Epochs { get; set; } |
Gets or sets the number of training epochs. |
int SelfDistillationGenerations { get; set; } |
Gets or sets the number of self-distillation generations (if using self-distillation). |
int? OutputDimension { get; set; } |
Gets or sets output dimension for models (if not inferrable from teacher). |
string CheckpointDirectory { get; set; } |
Gets or sets checkpoint directory path (if checkpoints are enabled). |
Methods
| Method | Summary |
|---|---|
void Validate() |
Validates the options and throws if any are invalid. |