AccuracyBasedAdaptiveStrategy<T> |
Adaptive distillation strategy that adjusts temperature based on student accuracy. |
AdaptiveTeacherModel<T> |
Adaptive teacher model that wraps a base teacher and provides its logits. |
AttentionDistillationStrategy<T> |
Implements attention-based knowledge distillation for transformer models. |
CheckpointMetadata |
Metadata about a saved checkpoint. |
ConfidenceBasedAdaptiveStrategy<T> |
Adaptive distillation strategy that adjusts temperature based on student confidence. |
ContrastiveDistillationStrategy<T> |
Implements Contrastive Representation Distillation (CRD) which transfers knowledge through contrastive learning of sample relationships rather than just matching outputs. |
CurriculumTeacherModel<T> |
Curriculum teacher that wraps a base teacher for curriculum learning scenarios. |
DistillationCheckpointManager<T> |
Manages checkpointing during knowledge distillation training. |
DistillationForwardResult<T> |
Encapsulates the result of a forward pass during knowledge distillation training. |
DistillationLossAdapter<T> |
Presents an IDistillationStrategy as a tape-capable loss so the standard gradient path can train a student with distillation. |
DistillationLoss<T> |
Implements the standard knowledge distillation loss function (Hinton et al., 2015). |
DistributedTeacherModel<T> |
Distributed teacher model that aggregates predictions from multiple distributed workers. |
EasyToHardCurriculumStrategy<T> |
Curriculum distillation strategy that progresses from easy to hard samples. |
EnsembleTeacherModel<T> |
Ensemble teacher model that combines predictions from multiple teacher models. |
EntropyBasedAdaptiveStrategy<T> |
Adaptive distillation strategy that adjusts temperature based on prediction entropy. |
FactorTransferDistillationStrategy<T> |
Factor transfer distillation that transfers knowledge through factorized representations. |
FeatureDistillationStrategy<T> |
Implements feature-based knowledge distillation (FitNets) where the student learns to match the teacher’s intermediate layer representations. |
FlowBasedDistillationStrategy<T> |
Flow-based distillation that matches the information flow between layers. |
HardToEasyCurriculumStrategy<T> |
Curriculum distillation strategy that progresses from hard to easy samples. |
HybridDistillationStrategy<T> |
Hybrid distillation strategy that combines multiple distillation strategies with configurable weights. |
IntermediateActivations<T> |
Stores intermediate layer activations collected during a forward pass. |
KnowledgeDistillationTrainer<T> |
Standard knowledge distillation trainer that uses a fixed teacher model to train a student. |
MultiModalTeacherModel<T> |
Multi-modal teacher that combines multiple input modalities (vision, text, audio). |
NeuronSelectivityDistillationStrategy<T> |
Neuron selectivity distillation that transfers the activation patterns and selectivity of individual neurons. |
OnlineTeacherModel<T> |
Online teacher model that updates its parameters during student training. |
PretrainedTeacherModel<T> |
Pretrained teacher model from external source (e.g., ImageNet, BERT). |
ProbabilisticDistillationStrategy<T> |
Probabilistic distillation that transfers distributional knowledge by matching statistical properties. |
QuantizedTeacherModel<T> |
Quantized teacher model with reduced precision for efficient deployment. |
RelationalDistillationStrategy<T> |
Implements Relational Knowledge Distillation (RKD) which transfers knowledge about relationships between samples rather than individual sample predictions. |
SelfDistillationTrainer<T> |
Implements self-distillation where a model acts as its own teacher to improve calibration and generalization. |
SelfTeacherModel<T> |
Self teacher model that uses the student’s own predictions from earlier training. |
SimilarityPreservingStrategy<T> |
Similarity-preserving distillation that preserves pairwise similarity structure. |
TeacherModelWrapper<T> |
Wraps an existing trained IFullModel to act as a teacher for knowledge distillation. |
TransformerTeacherModel<T> |
Transformer-based teacher model that provides logits from transformer architectures. |
VariationalDistillationStrategy<T> |
Variational distillation based on variational inference principles and information theory. |