KnowledgeDistillationOptions<T, TInput, TOutput>

Options & Configuration · AiDotNet.Models.Options

Configuration options for knowledge distillation training.

Inheritance: ModelOptions → KnowledgeDistillationOptions<T, TInput, TOutput>

For Beginners

This class configures how knowledge distillation works. Think of it as the “settings” for transferring knowledge from a large teacher model to a smaller student model.

How It Works

Quick Start Example:

Constructors

Constructor Summary
KnowledgeDistillationOptions()

Properties

Property Summary
Action<int, T> OnEpochComplete { get; set; } Gets or sets callback function invoked after each epoch.
Func<TInput, TOutput> TeacherForward { get; set; } Gets or sets the teacher model forward function (alternative approach).
IDistillationStrategy<T> Strategy { get; set; } Gets or sets the distillation strategy.
IFullModel<T, TInput, TOutput> TeacherModel { get; set; } Gets or sets the teacher IFullModel (recommended approach).
ITeacherModel<TInput, TOutput> Teacher { get; set; } Gets or sets the teacher model instance (if using pre-instantiated teacher).
TInput[] ValidationInputs { get; set; } Gets or sets validation data inputs (if validation is enabled).
TOutput[] ValidationLabels { get; set; } Gets or sets validation data labels (if validation is enabled).
TeacherModelType TeacherModelType { get; set; } Gets or sets the type of teacher model to use.
Vector<ITeacherModel<TInput, TOutput>> Teachers { get; set; } Gets or sets multiple teacher models (for ensemble distillation).
Vector<double> EnsembleWeights { get; set; } Gets or sets ensemble weights (if using multiple teachers).
Vector<string> AttentionLayers { get; set; } Gets or sets attention layer names (if using attention-based distillation).
Vector<string> FeatureLayerPairs { get; set; } Gets or sets layer pairs for feature-based distillation.
bool FreezeTeacher { get; set; } Gets or sets whether to freeze teacher model during training.
bool SaveCheckpoints { get; set; } Gets or sets whether to save checkpoints during training.
bool SaveOnlyBestCheckpoint { get; set; } Gets or sets whether to only save the best model checkpoint.
bool UseEMA { get; set; } Gets or sets whether to use exponential moving average for teacher predictions (self-distillation).
bool UseEarlyStopping { get; set; } Gets or sets whether to enable early stopping based on validation loss.
bool UseLabelSmoothing { get; set; } Gets or sets whether to use label smoothing.
bool ValidateAfterEpoch { get; set; } Gets or sets whether to validate model after each epoch.
double Alpha { get; set; } Gets or sets the alpha parameter balancing hard and soft loss.
double AttentionWeight { get; set; } Gets or sets weight for attention loss (if using attention-based distillation).
double EMADecay { get; set; } Gets or sets the EMA decay rate (if using EMA).
double EarlyStoppingMinDelta { get; set; } Gets or sets minimum improvement delta for early stopping.
double FeatureWeight { get; set; } Gets or sets weight for feature loss (if using feature-based distillation).
double LabelSmoothingFactor { get; set; } Gets or sets the label smoothing factor (if enabled).
double LearningRate { get; set; } Gets or sets the learning rate for student training.
double Temperature { get; set; } Gets or sets the temperature for softmax scaling.
int BatchSize { get; set; } Gets or sets the batch size for training.
int CheckpointFrequency { get; set; } Gets or sets checkpoint frequency (save every N epochs).
int EarlyStoppingPatience { get; set; } Gets or sets patience for early stopping (epochs without improvement).
int Epochs { get; set; } Gets or sets the number of training epochs.
int SelfDistillationGenerations { get; set; } Gets or sets the number of self-distillation generations (if using self-distillation).
int? OutputDimension { get; set; } Gets or sets output dimension for models (if not inferrable from teacher).
string CheckpointDirectory { get; set; } Gets or sets checkpoint directory path (if checkpoints are enabled).

Methods

Method Summary
void Validate() Validates the options and throws if any are invalid.