PMFAlgorithm<T, TInput, TOutput>
Models & Types · AiDotNet.MetaLearning.Algorithms
Implementation of PMF (P>M>F: Pre-training, Meta-training, Fine-tuning).
Inheritance: ModelBase<T, TInput, TOutput> → MetaLearnerBase<T, TInput, TOutput> → PMFAlgorithm<T, TInput, TOutput>
Implements: IGeneratedParameterRegistrar<T>
For Beginners
PMF combines the best of three worlds:
Pre-training: Learn general features from lots of data (foundation) Meta-training: Learn to adapt quickly from few examples (specialization) Fine-tuning: Squeeze out extra accuracy on each test task (polish)
This simple pipeline achieves remarkable results because each stage addresses a different aspect of few-shot learning.
How It Works
PMF implements a three-stage training pipeline:
- Pre-training: the feature extractor arrives pretrained (the model passed in as
MetaModel). - Meta-training: episodic training of that extractor as a ProtoNet with a cosine classifier. Each class’s
prototype is the mean of its support features, and the logits of a query are its cosine similarity to each
prototype divided by a learnable temperature (Temperature).
- Fine-tuning: optional per-task fine-tuning of the extractor on the support set with the same ProtoNet loss,
before the task’s prototypes are taken.
Meta-training used to be first-order MAML: inner gradient steps on the configured loss of the extractor’s raw
output, which never formed a prototype, and DistanceMetric was a
string nothing read.
Reference: Hu, S.X., Li, D., Stuhmer, J., Kim, M., & Hospedales, T.M. (2022). Pushing the Limits of Simple Pipelines for Few-Shot Learning. ICLR 2022.
Constructors
| Constructor | Summary |
|---|---|
PMFAlgorithm(PMFOptions<T, TInput, TOutput> options) |
Initializes a new PMF meta-learner. |
Properties
| Property | Summary |
|---|---|
MetaLearningAlgorithmType AlgorithmType { get; } |
Methods
| Method | Summary |
|---|---|
IModel<TInput, TOutput, ModelMetadata<T>> Adapt(IMetaLearningTask<T, TInput, TOutput> task) |
Adapts to a new task (Stage 3: F): optionally fine-tunes the extractor on the support set with the ProtoNet loss, then classifies by the support set’s prototypes. |
T MetaTrain(TaskBatch<T, TInput, TOutput> taskBatch) |
Performs one meta-training step (Stage 2: M): the ProtoNet loss of each episode’s query examples, differentiated through the prototypes into the feature extractor and into the cosine scale. |