DKTAlgorithm<T, TInput, TOutput>
Models & Types · AiDotNet.MetaLearning.Algorithms
Implementation of DKT (Deep Kernel Transfer) (Patacchiola et al., ICLR 2020).
Inheritance: ModelBase<T, TInput, TOutput> → MetaLearnerBase<T, TInput, TOutput> → DKTAlgorithm<T, TInput, TOutput>
Implements: IGeneratedParameterRegistrar<T>
For Beginners
DKT pairs a neural network with Gaussian processes:
Standard approach: Neural network extracts features, then a simple classifier (e.g., nearest centroid) classifies.
DKT’s approach:
- Neural network extracts features (learns what to compare)
- GP kernel computes similarity in feature space (learns how to compare)
- GP provides predictions WITH uncertainty (knows when it’s uncertain)
Why Gaussian Processes?
- Principled uncertainty: “I’m 90% confident it’s a cat” vs “I have no idea”
- Non-parametric: Adapts to any number of support examples
- Kernel-based: The deep kernel captures complex, learned similarities
The deep kernel: Instead of a fixed kernel (like RBF), the kernel operates on learned features: k(x, x’) = k_base(f(x), f(x’)) where f is the neural network and k_base is a standard kernel. Both are trained end-to-end.
How It Works
DKT combines deep feature extractors with Gaussian processes. The neural network learns a feature space in which a GP classifier provides principled Bayesian predictions with uncertainty estimates.
Algorithm - DKT:
Implementation Notes:
- MetaTrain uses classification loss as a pragmatic approximation of the GP marginal
log-likelihood. The full GP-based optimization requires matrix inversions per task which is expensive; the classification loss provides equivalent gradient signal for training the feature extractor end-to-end.
- The modulation strategy computes a single scalar from GP kernel weights and applies it
uniformly to all backbone parameters. This is a deliberate simplification; per-layer or per-feature-group modulation could better preserve the GP’s learned relationships but would require additional complexity.
- Adapted models (DKTModel) mutate the shared model instance via SetParameters in Predict.
Callers must ensure single-threaded access to each adapted model instance.
Reference: Patacchiola, M., Turner, J., Crowley, E.J., O’Boyle, M., & Sherron, A. (2020). Bayesian Meta-Learning for the Few-Shot Setting via Deep Kernels. ICLR 2020.
Constructors
| Constructor | Summary |
|---|---|
DKTAlgorithm(DKTOptions<T, TInput, TOutput> options) |
Initializes a new DKT meta-learner. |
Properties
| Property | Summary |
|---|---|
MetaLearningAlgorithmType AlgorithmType { get; } |
Methods
| Method | Summary |
|---|---|
IModel<TInput, TOutput, ModelMetadata<T>> Adapt(IMetaLearningTask<T, TInput, TOutput> task) |
|
T MetaTrain(TaskBatch<T, TInput, TOutput> taskBatch) |