PMFAlgorithm<T, TInput, TOutput>

Models & Types · AiDotNet.MetaLearning.Algorithms

Implementation of PMF (P>M>F: Pre-training, Meta-training, Fine-tuning).

Inheritance: ModelBase<T, TInput, TOutput> → MetaLearnerBase<T, TInput, TOutput> → PMFAlgorithm<T, TInput, TOutput>

Implements: IGeneratedParameterRegistrar<T>

For Beginners

PMF combines the best of three worlds:

Pre-training: Learn general features from lots of data (foundation) Meta-training: Learn to adapt quickly from few examples (specialization) Fine-tuning: Squeeze out extra accuracy on each test task (polish)

This simple pipeline achieves remarkable results because each stage addresses a different aspect of few-shot learning.

How It Works

PMF implements a three-stage training pipeline:

  1. Pre-training: the feature extractor arrives pretrained (the model passed in as MetaModel).
  2. Meta-training: episodic training of that extractor as a ProtoNet with a cosine classifier. Each class’s

prototype is the mean of its support features, and the logits of a query are its cosine similarity to each prototype divided by a learnable temperature (Temperature).

  1. Fine-tuning: optional per-task fine-tuning of the extractor on the support set with the same ProtoNet loss,

before the task’s prototypes are taken.

Meta-training used to be first-order MAML: inner gradient steps on the configured loss of the extractor’s raw output, which never formed a prototype, and DistanceMetric was a string nothing read.

Reference: Hu, S.X., Li, D., Stuhmer, J., Kim, M., & Hospedales, T.M. (2022). Pushing the Limits of Simple Pipelines for Few-Shot Learning. ICLR 2022.

Constructors

Constructor Summary
PMFAlgorithm(PMFOptions<T, TInput, TOutput> options) Initializes a new PMF meta-learner.

Properties

Property Summary
MetaLearningAlgorithmType AlgorithmType { get; }

Methods

Method Summary
IModel<TInput, TOutput, ModelMetadata<T>> Adapt(IMetaLearningTask<T, TInput, TOutput> task) Adapts to a new task (Stage 3: F): optionally fine-tunes the extractor on the support set with the ProtoNet loss, then classifies by the support set’s prototypes.
T MetaTrain(TaskBatch<T, TInput, TOutput> taskBatch) Performs one meta-training step (Stage 2: M): the ProtoNet loss of each episode’s query examples, differentiated through the prototypes into the feature extractor and into the cosine scale.