AWGIMAlgorithm<T, TInput, TOutput>
Models & Types · AiDotNet.MetaLearning.Algorithms
AWGIM: Attentive Weights Generation for few-shot learning via Information Maximization (Guo & Cheung, CVPR 2020).
Inheritance: ModelBase<T, TInput, TOutput> → MetaLearnerBase<T, TInput, TOutput> → AWGIMAlgorithm<T, TInput, TOutput>
Implements: IGeneratedParameterRegistrar<T>
For Beginners
Most meta-learners adapt a model to a new task by taking a few training steps on the examples. AWGIM instead writes the classifier directly: it reads the task’s examples and the example to classify, and outputs the classifier’s weights. Because it writes a separate classifier for each example it classifies, it can focus on whichever support examples matter for that query.
How It Works
AWGIM generates a linear classifier for every query example in one forward pass, with no inner
optimization loop. LEO, which it builds on, does run one. A contextual path (self-attention over the
support set) and an attentive path (the query attending to the support set) are decoded into a weight
distribution per (query, support row) pair. Averaging over each class’s rows gives the query its own
classifier. See AwgimNetwork for the generator.
Objective. L = CE(query) + α1·CE(support, classified by every query’s weights)
- α2·‖r_c(w) − sg(contextual code)‖² + α3·‖r_q(w) − sg(attentive code)‖². The two reconstruction terms are the variational surrogates of the mutual information between the generated weights and the support set and the query. They keep the weights from discarding what the two paths computed.
Meta-update. AdamW, as the authors train it: decoupled weight decay, a staircase learning-rate decay, and gradients clipped first by value, then by each tensor’s norm. Should the loss or a gradient be non-finite, the step instead follows the gradient of the kernels’ L2 penalty, the reference’s recovery rule. The feature encoder receives the same objective’s gradient through its embeddings and is updated by the same rule.
Constructors
| Constructor | Summary |
|---|---|
AWGIMAlgorithm(AWGIMOptions<T, TInput, TOutput> options) |
Creates AWGIM over the feature encoder in options. |
Properties
| Property | Summary |
|---|---|
MetaLearningAlgorithmType AlgorithmType { get; } |
Methods
| Method | Summary |
|---|---|
IModel<TInput, TOutput, ModelMetadata<T>> Adapt(IMetaLearningTask<T, TInput, TOutput> task) |
|
ModelOptions GetOptions() |
|
T MetaTrain(TaskBatch<T, TInput, TOutput> taskBatch) |