DistillationStrategyFactory<T>
Helpers & Utilities · AiDotNet.KnowledgeDistillation
Factory of named knowledge-distillation strategies.
How It Works
Each named strategy is a public factory method returning an
IDistillationStrategy. The strategy IS the parameter: callers pass a
strategy instance directly (e.g. via KnowledgeDistillationOptions.Strategy or
ConfigureDistillationStrategy). When no strategy is supplied, response-based
distillation (Hinton et al., 2015) is the industry-standard default — see
ResolveStrategy.
Methods
| Method | Summary |
|---|---|
IDistillationStrategy<T> CreateAttentionBasedStrategy(Vector<string> attentionLayers, double temperature, double alpha, double attentionWeight) |
Attention-transfer distillation (Zagoruyko & Komodakis, 2017), for transformer models. |
IDistillationStrategy<T> CreateContrastiveStrategy(double temperature, double alpha, ContrastiveMode mode) |
Contrastive Representation Distillation / CRD (Tian et al., 2020). |
IDistillationStrategy<T> CreateFactorTransferStrategy(double temperature, double alpha) |
Factor Transfer (Kim et al., 2018). |
IDistillationStrategy<T> CreateFeatureBasedStrategy(Vector<string> layerPairs, double featureWeight) |
Feature-based / FitNets distillation (Romero et al., 2014). |
IDistillationStrategy<T> CreateFlowBasedStrategy(double temperature, double alpha) |
Flow of Solution Procedure / FSP (Yim et al., 2017). |
IDistillationStrategy<T> CreateHybridStrategy(double temperature, double alpha, Vector<IDistillationStrategy<T>> strategies, Vector<double> strategyWeights) |
Combined/hybrid distillation. |
IDistillationStrategy<T> CreateNeuronSelectivityStrategy(double temperature, double alpha) |
Neuron Selectivity Transfer / NST (Huang & Wang, 2017). |
IDistillationStrategy<T> CreateProbabilisticStrategy(double temperature, double alpha) |
Probabilistic Knowledge Transfer / PKT (Passalis & Tefas, 2018). |
IDistillationStrategy<T> CreateRelationBasedStrategy(double temperature, double alpha) |
Relational Knowledge Distillation / RKD (Park et al., 2019). |
IDistillationStrategy<T> CreateResponseBasedStrategy(double temperature, double alpha) |
Response-based distillation (Hinton et al., 2015) — the industry-standard default. |
IDistillationStrategy<T> CreateSimilarityPreservingStrategy(double temperature, double alpha) |
Similarity-Preserving Distillation / SP (Tung & Mori, 2019). |
IDistillationStrategy<T> CreateVariationalStrategy(double temperature, double alpha) |
Variational Information Distillation / VID (Ahn et al., 2019). |
IDistillationStrategy<T> ResolveStrategy(IDistillationStrategy<T> strategy, double temperature, double alpha) |
Returns strategy when non-null, otherwise the response-based default (standard Hinton distillation) configured with the supplied temperature and alpha. |