ANILAlgorithm<T, TInput, TOutput>
Models & Types · AiDotNet.MetaLearning.Algorithms
Implementation of Almost No Inner Loop (ANIL) meta-learning algorithm.
Inheritance: ModelBase<T, TInput, TOutput> → MetaLearnerBase<T, TInput, TOutput> → ANILAlgorithm<T, TInput, TOutput>
Implements: IGeneratedParameterRegistrar<T>
For Beginners
Think of a neural network as having two parts: a body that turns each example into a description (an embedding), and a head that turns a description into a score for each class. ANIL keeps the body fixed while it adapts to a new task and only retrains the small head; the body improves across tasks.
How It Works
ANIL (Raghu et al. 2020) is MAML with the inner loop removed from every layer but the head: each task adapts only the final linear classifier on its support set, while the body (the meta-model, which embeds each example) is updated only by the outer loop. The outer loop meta-learns the body and the head’s initialisation together from the query loss after adaptation.
Per example. The body embeds every row of the input into a FeatureDimension-wide
representation, and the head scores every row: logits = h W^T + b, one row of
NumClasses scores per example. The loss is the configured loss on
those logits against one class index per example - cross-entropy by default, the paper’s classification loss.
Exact meta-gradient. “We do not remove second order terms in ANIL (unlike in first-order MAML); second
order terms still persist through the derivative of the inner loop update for the head parameters” (Raghu et
al. 2020, App. C.1). The head’s initialisation receives prod_k (I - alpha H_k) applied to the query
gradient, and the body receives, besides its direct query gradient, the cross term through each inner step’s
support gradient. Both second-order pieces are Hessian-vector products taken as central differences of
gradients along the head direction, so they cost two gradient evaluations per inner step.
UseFirstOrder drops them.
Reference: Raghu, A., Raghu, M., Bengio, S., & Vinyals, O. (2020). Rapid Learning or Feature Reuse? Towards Understanding the Effectiveness of MAML.
Constructors
| Constructor | Summary |
|---|---|
ANILAlgorithm(ANILOptions<T, TInput, TOutput> options) |
Initializes a new instance of the ANILAlgorithm class. |
Properties
| Property | Summary |
|---|---|
MetaLearningAlgorithmType AlgorithmType { get; } |
Gets the algorithm type identifier for this meta-learner. |
Methods
| Method | Summary |
|---|---|
IModel<TInput, TOutput, ModelMetadata<T>> Adapt(IMetaLearningTask<T, TInput, TOutput> task) |
Adapts the meta-learned model to a new task by only updating the classification head. |
T MetaTrain(TaskBatch<T, TInput, TOutput> taskBatch) |
Performs one meta-training step: head-only adaptation per task, then one update of the body and the head’s initialisation from the query losses. |