ATAMLAlgorithm<T, TInput, TOutput>

Models & Types · AiDotNet.MetaLearning.Algorithms

Implementation of ATAML: Attentive Task-Agnostic Meta-Learning.

Inheritance: ModelBase<T, TInput, TOutput> → MetaLearnerBase<T, TInput, TOutput> → ATAMLAlgorithm<T, TInput, TOutput>

Implements: IGeneratedParameterRegistrar<T>

For Beginners

ATAML learns one shared “reader” that stays fixed for every task, plus a small per-task part that decides which bits of what was read matter and turns them into a class. Only the small part changes when it meets a new task.

How It Works

Jiang et al. 2018 split a MAML-style learner in two. The shared parameters theta_E - the encoder - learn a task-agnostic representation and are never adapted per task. The task-specific parameters theta_T = {theta_ATT, theta_W} - a content-based attention vector and a softmax classifier - are the only ones the inner loop touches. The meta-update then moves both, through the loss measured AFTER adaptation (Algorithm 1, lines 5-8).

The attentive base learner (eq. 4-6). The encoder maps an input to states s_t. Each state is scored by an inner product with the attention vector, alpha_t = theta_ATT . s_t, which rescales it, and the rescaled states are averaged into one context vector: c = mean_t(alpha_t s_t). The context is classified by softmax(c; theta_W). Attention here weights the ENCODER’S REPRESENTATION - in the paper, the words of a document.

What this replaced. The previous implementation bucketed the GRADIENT into fixed-size chunks, took a softmax over those buckets, and used the result as per-parameter learning-rate multipliers while adapting EVERY parameter. That is a learned step size - essentially Meta-SGD - not attention over a representation, and it had neither the shared/task-specific split nor a classifier head. The attention projection was also trained by simultaneous perturbation. None of that is in the paper.

Reference: Jiang, X., Havaei, M., Chartrand, G., et al. (2018). On the Importance of Attention in Meta-Learning for Few-Shot Text Classification.

Constructors

Constructor Summary
ATAMLAlgorithm(ATAMLOptions<T, TInput, TOutput> options) Initializes a new ATAML meta-learner.

Properties

Property Summary
MetaLearningAlgorithmType AlgorithmType { get; }

Methods

Method Summary
IModel<TInput, TOutput, ModelMetadata<T>> Adapt(IMetaLearningTask<T, TInput, TOutput> task)
ModelOptions GetOptions()
T MetaTrain(TaskBatch<T, TInput, TOutput> taskBatch) Algorithm 1: adapt theta_T on each task’s support set, measure the query loss after adaptation, and move the shared encoder and the task parameters’ initialisation by that loss.