TADAMAlgorithm<T, TInput, TOutput>

Models & Types · AiDotNet.MetaLearning.Algorithms

Implementation of Task-Dependent Adaptive Metric (TADAM) algorithm for few-shot learning.

Inheritance: ModelBase<T, TInput, TOutput> → MetaLearnerBase<T, TInput, TOutput> → TADAMAlgorithm<T, TInput, TOutput>

Implements: IGeneratedParameterRegistrar<T>

For Beginners

TADAM improves on ProtoNets by making the feature extractor “aware” of the current task:

How it works:

  1. Extract features from support set examples
  2. Summarize the task as the average of its class prototypes
  3. Use that summary to rescale each feature dimension for this task
  4. Compute prototypes from the rescaled features
  5. Classify queries by their scaled distance to each prototype

How It Works

TADAM (Oreshkin, Rodriguez and Lacoste, NeurIPS 2018) extends Prototypical Networks with:

  1. Metric scaling (Sec. 2.1, Eq. 3): p(y = k | x) = softmax(-alpha d(f(x), c_k)) with ONE learned scalar

alpha - not a per-dimension weight vector and not a separate temperature, which alpha already is.

  1. Task conditioning (Sec. 2.2): a task-embedding network (TEN) maps the task representation c_bar, the

mean of the class prototypes, to FiLM parameters, with a scalar post-multiplier that starts at zero and is L2-penalized, so an untrained TEN is the identity.

  1. Auxiliary co-training (Sec. 2.3), here an extra classification term on the support set.

The feature extractor is a black-box model, so FiLM is applied to its OUTPUT features rather than inside its layers: h' = (1 + gamma) h. A shift beta would cancel exactly in every query-prototype difference at that level, so only the scale is generated. The extractor, alpha and the TEN all train on the exact gradient of the episode’s query cross-entropy: the logits are built with engine tensor ops, so the extractor’s gradient flows through the prototypes and the metric (it was trained against the raw labels before).

Algorithm - TADAM:

Constructors

Constructor Summary
TADAMAlgorithm(TADAMOptions<T, TInput, TOutput> options) Initializes a new instance of the TADAMAlgorithm class.

Properties

Property Summary
MetaLearningAlgorithmType AlgorithmType { get; }

Methods

Method Summary
IModel<TInput, TOutput, ModelMetadata<T>> Adapt(IMetaLearningTask<T, TInput, TOutput> task)
T MetaTrain(TaskBatch<T, TInput, TOutput> taskBatch)