PMFOptions<T, TInput, TOutput>
Options & Configuration · AiDotNet.MetaLearning.Options
Configuration options for PMF (P>M>F: Pre-training, Meta-training, Fine-tuning) (Hu et al., ICLR 2022).
Inheritance: ModelOptions → PMFOptions<T, TInput, TOutput>
Implements: IMetaLearnerOptions<T>
For Beginners
PMF is a three-step recipe for few-shot learning:
Stage 1: Pre-training (P) Train a powerful feature extractor on a large dataset (like ImageNet). This gives a strong foundation of general visual knowledge.
Stage 2: Meta-training (M) Fine-tune the feature extractor episodically as a ProtoNet with a cosine classifier: each class’s prototype is the mean of its support features, and a query’s logits are its cosine similarity to each prototype divided by a learnable temperature. This adapts the pretrained features for few-shot scenarios.
Stage 3: Fine-tuning (F) Optionally fine-tune on each test task’s support set, with the same ProtoNet loss, for extra performance.
Why it works: Pre-training provides rich, transferable features. Meta-training adapts them for few-shot scenarios. Fine-tuning squeezes out the last bit of accuracy. Each stage builds on the previous one.
How It Works
PMF introduces a three-stage training pipeline that leverages large-scale pretraining before meta-learning. The key insight is that combining pre-training with meta-training and optional fine-tuning achieves state-of-the-art few-shot performance.
Reference: Hu, S.X., Li, D., Stuhmer, J., Kim, M., & Hospedales, T.M. (2022). Pushing the Limits of Simple Pipelines for Few-Shot Learning. ICLR 2022.
Constructors
| Constructor | Summary |
|---|---|
PMFOptions(IFullModel<T, TInput, TOutput> metaModel) |
Initializes a new instance of PMFOptions. |
Properties
| Property | Summary |
|---|---|
IEpisodicDataLoader<T, TInput, TOutput> DataLoader { get; set; } |
Gets or sets the episodic data loader. |
IFullModel<T, TInput, TOutput> MetaModel { get; set; } |
Gets or sets the feature extractor model. |
IGradientBasedOptimizer<T, TInput, TOutput> InnerOptimizer { get; set; } |
Optimizer for task adaptation. |
IGradientBasedOptimizer<T, TInput, TOutput> MetaOptimizer { get; set; } |
Optimizer for the meta-update. |
ILossFunction<T> LossFunction { get; set; } |
Loss function for meta-training; null uses the default multi-class classification loss. |
PMFDistanceMetric DistanceMetric { get; set; } |
Gets or sets the metric the ProtoNet classifier of Stage 2 (M) and of adaptation scores queries with. |
bool EnableCheckpointing { get; set; } |
|
bool EnableFineTuning { get; set; } |
Gets or sets whether to use fine-tuning during adaptation (Stage 3). |
bool LearnTemperature { get; set; } |
Gets or sets whether the cosine classifier’s scale 1 / Temperature is learned during meta-training, as in the paper’s ProtoNet, or kept fixed at its initial value. |
bool UseFirstOrder { get; set; } |
|
double FineTuningLearningRate { get; set; } |
Gets or sets the fine-tuning learning rate for Stage 3 (F). |
double InnerLearningRate { get; set; } |
|
double OuterLearningRate { get; set; } |
Learning rate of the outer (meta) update. |
double Temperature { get; set; } |
Gets or sets the initial temperature of the cosine classifier: the logits are cos(f(x), c_k) / Temperature. |
double? GradientClipThreshold { get; set; } |
|
int AdaptationSteps { get; set; } |
|
int CheckpointFrequency { get; set; } |
|
int EvaluationFrequency { get; set; } |
|
int EvaluationTasks { get; set; } |
|
int FineTuningSteps { get; set; } |
Gets or sets the number of fine-tuning steps during adaptation. |
int MetaBatchSize { get; set; } |
|
int NumMetaIterations { get; set; } |
|
int? RandomSeed { get; set; } |
Gets or sets the random seed. |
Methods
| Method | Summary |
|---|---|
IMetaLearnerOptions<T> Clone() |
|
bool IsValid() |