PMFOptions<T, TInput, TOutput>

Options & Configuration · AiDotNet.MetaLearning.Options

Configuration options for PMF (P>M>F: Pre-training, Meta-training, Fine-tuning) (Hu et al., ICLR 2022).

Inheritance: ModelOptions → PMFOptions<T, TInput, TOutput>

Implements: IMetaLearnerOptions<T>

For Beginners

PMF is a three-step recipe for few-shot learning:

Stage 1: Pre-training (P) Train a powerful feature extractor on a large dataset (like ImageNet). This gives a strong foundation of general visual knowledge.

Stage 2: Meta-training (M) Fine-tune the feature extractor episodically as a ProtoNet with a cosine classifier: each class’s prototype is the mean of its support features, and a query’s logits are its cosine similarity to each prototype divided by a learnable temperature. This adapts the pretrained features for few-shot scenarios.

Stage 3: Fine-tuning (F) Optionally fine-tune on each test task’s support set, with the same ProtoNet loss, for extra performance.

Why it works: Pre-training provides rich, transferable features. Meta-training adapts them for few-shot scenarios. Fine-tuning squeezes out the last bit of accuracy. Each stage builds on the previous one.

How It Works

PMF introduces a three-stage training pipeline that leverages large-scale pretraining before meta-learning. The key insight is that combining pre-training with meta-training and optional fine-tuning achieves state-of-the-art few-shot performance.

Reference: Hu, S.X., Li, D., Stuhmer, J., Kim, M., & Hospedales, T.M. (2022). Pushing the Limits of Simple Pipelines for Few-Shot Learning. ICLR 2022.

Constructors

Constructor Summary
PMFOptions(IFullModel<T, TInput, TOutput> metaModel) Initializes a new instance of PMFOptions.

Properties

Property Summary
IEpisodicDataLoader<T, TInput, TOutput> DataLoader { get; set; } Gets or sets the episodic data loader.
IFullModel<T, TInput, TOutput> MetaModel { get; set; } Gets or sets the feature extractor model.
IGradientBasedOptimizer<T, TInput, TOutput> InnerOptimizer { get; set; } Optimizer for task adaptation.
IGradientBasedOptimizer<T, TInput, TOutput> MetaOptimizer { get; set; } Optimizer for the meta-update.
ILossFunction<T> LossFunction { get; set; } Loss function for meta-training; null uses the default multi-class classification loss.
PMFDistanceMetric DistanceMetric { get; set; } Gets or sets the metric the ProtoNet classifier of Stage 2 (M) and of adaptation scores queries with.
bool EnableCheckpointing { get; set; }
bool EnableFineTuning { get; set; } Gets or sets whether to use fine-tuning during adaptation (Stage 3).
bool LearnTemperature { get; set; } Gets or sets whether the cosine classifier’s scale 1 / Temperature is learned during meta-training, as in the paper’s ProtoNet, or kept fixed at its initial value.
bool UseFirstOrder { get; set; }
double FineTuningLearningRate { get; set; } Gets or sets the fine-tuning learning rate for Stage 3 (F).
double InnerLearningRate { get; set; }
double OuterLearningRate { get; set; } Learning rate of the outer (meta) update.
double Temperature { get; set; } Gets or sets the initial temperature of the cosine classifier: the logits are cos(f(x), c_k) / Temperature.
double? GradientClipThreshold { get; set; }
int AdaptationSteps { get; set; }
int CheckpointFrequency { get; set; }
int EvaluationFrequency { get; set; }
int EvaluationTasks { get; set; }
int FineTuningSteps { get; set; } Gets or sets the number of fine-tuning steps during adaptation.
int MetaBatchSize { get; set; }
int NumMetaIterations { get; set; }
int? RandomSeed { get; set; } Gets or sets the random seed.

Methods

Method Summary
IMetaLearnerOptions<T> Clone()
bool IsValid()