AWGIMAlgorithm<T, TInput, TOutput>

Models & Types · AiDotNet.MetaLearning.Algorithms

AWGIM: Attentive Weights Generation for few-shot learning via Information Maximization (Guo & Cheung, CVPR 2020).

Inheritance: ModelBase<T, TInput, TOutput> → MetaLearnerBase<T, TInput, TOutput> → AWGIMAlgorithm<T, TInput, TOutput>

Implements: IGeneratedParameterRegistrar<T>

For Beginners

Most meta-learners adapt a model to a new task by taking a few training steps on the examples. AWGIM instead writes the classifier directly: it reads the task’s examples and the example to classify, and outputs the classifier’s weights. Because it writes a separate classifier for each example it classifies, it can focus on whichever support examples matter for that query.

How It Works

AWGIM generates a linear classifier for every query example in one forward pass, with no inner optimization loop. LEO, which it builds on, does run one. A contextual path (self-attention over the support set) and an attentive path (the query attending to the support set) are decoded into a weight distribution per (query, support row) pair. Averaging over each class’s rows gives the query its own classifier. See AwgimNetwork for the generator.

Objective. L = CE(query) + α1·CE(support, classified by every query’s weights)

  • α2·‖r_c(w) − sg(contextual code)‖² + α3·‖r_q(w) − sg(attentive code)‖². The two reconstruction terms are the variational surrogates of the mutual information between the generated weights and the support set and the query. They keep the weights from discarding what the two paths computed.

Meta-update. AdamW, as the authors train it: decoupled weight decay, a staircase learning-rate decay, and gradients clipped first by value, then by each tensor’s norm. Should the loss or a gradient be non-finite, the step instead follows the gradient of the kernels’ L2 penalty, the reference’s recovery rule. The feature encoder receives the same objective’s gradient through its embeddings and is updated by the same rule.

Constructors

Constructor Summary
AWGIMAlgorithm(AWGIMOptions<T, TInput, TOutput> options) Creates AWGIM over the feature encoder in options.

Properties

Property Summary
MetaLearningAlgorithmType AlgorithmType { get; }

Methods

Method Summary
IModel<TInput, TOutput, ModelMetadata<T>> Adapt(IMetaLearningTask<T, TInput, TOutput> task)
ModelOptions GetOptions()
T MetaTrain(TaskBatch<T, TInput, TOutput> taskBatch)