ATAMLAlgorithm<T, TInput, TOutput>
Models & Types · AiDotNet.MetaLearning.Algorithms
Implementation of ATAML: Attentive Task-Agnostic Meta-Learning.
Inheritance: ModelBase<T, TInput, TOutput> → MetaLearnerBase<T, TInput, TOutput> → ATAMLAlgorithm<T, TInput, TOutput>
Implements: IGeneratedParameterRegistrar<T>
For Beginners
ATAML learns one shared “reader” that stays fixed for every task, plus a small per-task part that decides which bits of what was read matter and turns them into a class. Only the small part changes when it meets a new task.
How It Works
Jiang et al. 2018 split a MAML-style learner in two. The shared parameters theta_E - the encoder -
learn a task-agnostic representation and are never adapted per task. The task-specific parameters
theta_T = {theta_ATT, theta_W} - a content-based attention vector and a softmax classifier - are the
only ones the inner loop touches. The meta-update then moves both, through the loss measured AFTER
adaptation (Algorithm 1, lines 5-8).
The attentive base learner (eq. 4-6). The encoder maps an input to states s_t. Each
state is scored by an inner product with the attention vector, alpha_t = theta_ATT . s_t, which
rescales it, and the rescaled states are averaged into one context vector:
c = mean_t(alpha_t s_t). The context is classified by softmax(c; theta_W). Attention here
weights the ENCODER’S REPRESENTATION - in the paper, the words of a document.
What this replaced. The previous implementation bucketed the GRADIENT into fixed-size chunks, took a softmax over those buckets, and used the result as per-parameter learning-rate multipliers while adapting EVERY parameter. That is a learned step size - essentially Meta-SGD - not attention over a representation, and it had neither the shared/task-specific split nor a classifier head. The attention projection was also trained by simultaneous perturbation. None of that is in the paper.
Reference: Jiang, X., Havaei, M., Chartrand, G., et al. (2018). On the Importance of Attention in Meta-Learning for Few-Shot Text Classification.
Constructors
| Constructor | Summary |
|---|---|
ATAMLAlgorithm(ATAMLOptions<T, TInput, TOutput> options) |
Initializes a new ATAML meta-learner. |
Properties
| Property | Summary |
|---|---|
MetaLearningAlgorithmType AlgorithmType { get; } |
Methods
| Method | Summary |
|---|---|
IModel<TInput, TOutput, ModelMetadata<T>> Adapt(IMetaLearningTask<T, TInput, TOutput> task) |
|
ModelOptions GetOptions() |
|
T MetaTrain(TaskBatch<T, TInput, TOutput> taskBatch) |
Algorithm 1: adapt theta_T on each task’s support set, measure the query loss after adaptation, and move the shared encoder and the task parameters’ initialisation by that loss. |