MAMLPlusPlusOptions<T, TInput, TOutput>

Options & Configuration · AiDotNet.MetaLearning.Options

Configuration options for MAML++ (How to Train Your MAML) meta-learning algorithm.

Inheritance: ModelOptions → MAMLPlusPlusOptions<T, TInput, TOutput>

Implements: IMetaLearnerOptions<T>

For Beginners

MAML++ is “MAML done right” for production use.

Original MAML has several practical problems:

  • Training can be unstable (loss explodes)
  • Batch normalization doesn’t work well
  • A single learning rate isn’t optimal for all inner steps
  • Second-order gradients are expensive and often unhelpful early on

MAML++ fixes ALL of these with:

  1. Multi-Step Loss: Supervise every inner step, not just the last one
  2. Per-Step Learning Rates: Each adaptation step has its own learning rate
  3. Derivative-Order Annealing: Start with first-order, gradually add second-order
  4. Stable Batch Norm: Per-step batch norm statistics for stable adaptation

Think of it like upgrading from a prototype to production code - same idea, but with all the engineering needed to work reliably at scale.

How It Works

MAML++ is a production-hardened version of MAML that addresses several training instabilities through multi-step loss optimization, derivative-order annealing, per-step learning rates, and batch normalization stability fixes.

Reference: Antoniou, A., Edwards, H., & Storkey, A. (2019). How to Train Your MAML. ICLR 2019.

Constructors

Constructor Summary
MAMLPlusPlusOptions(IFullModel<T, TInput, TOutput> metaModel) Initializes a new instance of the MAMLPlusPlusOptions class.

Properties

Property Summary
IEpisodicDataLoader<T, TInput, TOutput> DataLoader { get; set; } Gets or sets the episodic data loader for sampling tasks.
IFullModel<T, TInput, TOutput> MetaModel { get; set; } Gets or sets the meta-model to be trained.
IGradientBasedOptimizer<T, TInput, TOutput> InnerOptimizer { get; set; } Optimizer for task adaptation.
IGradientBasedOptimizer<T, TInput, TOutput> MetaOptimizer { get; set; } Optimizer for the meta-update.
ILossFunction<T> LossFunction { get; set; } Loss function for meta-training; null uses the default multi-class classification loss.
bool EnableCheckpointing { get; set; } Gets or sets whether to save checkpoints.
bool UseDerivativeOrderAnnealing { get; set; } Gets or sets whether to use derivative-order annealing.
bool UseFirstOrder { get; set; } Gets or sets whether to use first-order approximation.
bool UseMultiStepLoss { get; set; } Gets or sets whether to use multi-step loss optimization (MSL).
bool UsePerStepBatchNorm { get; set; } Gets or sets whether to use per-step batch normalization statistics.
bool UsePerStepLearningRates { get; set; } Gets or sets whether to use per-step learning rates (LSLR).
double CosineAnnealingMinRatio { get; set; } Gets or sets the cosine annealing schedule minimum learning rate ratio.
double InnerLearningRate { get; set; } Gets or sets the base inner learning rate (used for initialization of per-step rates).
double OuterLearningRate { get; set; } Gets or sets the outer learning rate for meta-parameter updates.
double? GradientClipThreshold { get; set; } Gets or sets the gradient clipping threshold.
double[] MultiStepLossWeights { get; set; } Gets or sets the weights for multi-step loss at each adaptation step.
int AdaptationSteps { get; set; } Gets or sets the number of inner-loop adaptation steps.
int AnnealingIterations { get; set; } Gets or sets the number of iterations over which to anneal from first-order to second-order.
int CheckpointFrequency { get; set; } Gets or sets the checkpoint frequency.
int EvaluationFrequency { get; set; } Gets or sets the evaluation frequency.
int EvaluationTasks { get; set; } Gets or sets the number of evaluation tasks.
int MetaBatchSize { get; set; } Gets or sets the number of tasks per meta-batch.
int NumMetaIterations { get; set; } Gets or sets the total number of meta-training iterations.
int? RandomSeed { get; set; } Gets or sets the random seed.

Methods

Method Summary
IMetaLearnerOptions<T> Clone()
bool IsValid()