MAMLPlusPlusOptions<T, TInput, TOutput>
Options & Configuration · AiDotNet.MetaLearning.Options
Configuration options for MAML++ (How to Train Your MAML) meta-learning algorithm.
Inheritance: ModelOptions → MAMLPlusPlusOptions<T, TInput, TOutput>
Implements: IMetaLearnerOptions<T>
For Beginners
MAML++ is “MAML done right” for production use.
Original MAML has several practical problems:
- Training can be unstable (loss explodes)
- Batch normalization doesn’t work well
- A single learning rate isn’t optimal for all inner steps
- Second-order gradients are expensive and often unhelpful early on
MAML++ fixes ALL of these with:
- Multi-Step Loss: Supervise every inner step, not just the last one
- Per-Step Learning Rates: Each adaptation step has its own learning rate
- Derivative-Order Annealing: Start with first-order, gradually add second-order
- Stable Batch Norm: Per-step batch norm statistics for stable adaptation
Think of it like upgrading from a prototype to production code - same idea, but with all the engineering needed to work reliably at scale.
How It Works
MAML++ is a production-hardened version of MAML that addresses several training instabilities through multi-step loss optimization, derivative-order annealing, per-step learning rates, and batch normalization stability fixes.
Reference: Antoniou, A., Edwards, H., & Storkey, A. (2019). How to Train Your MAML. ICLR 2019.
Constructors
| Constructor | Summary |
|---|---|
MAMLPlusPlusOptions(IFullModel<T, TInput, TOutput> metaModel) |
Initializes a new instance of the MAMLPlusPlusOptions class. |
Properties
| Property | Summary |
|---|---|
IEpisodicDataLoader<T, TInput, TOutput> DataLoader { get; set; } |
Gets or sets the episodic data loader for sampling tasks. |
IFullModel<T, TInput, TOutput> MetaModel { get; set; } |
Gets or sets the meta-model to be trained. |
IGradientBasedOptimizer<T, TInput, TOutput> InnerOptimizer { get; set; } |
Optimizer for task adaptation. |
IGradientBasedOptimizer<T, TInput, TOutput> MetaOptimizer { get; set; } |
Optimizer for the meta-update. |
ILossFunction<T> LossFunction { get; set; } |
Loss function for meta-training; null uses the default multi-class classification loss. |
bool EnableCheckpointing { get; set; } |
Gets or sets whether to save checkpoints. |
bool UseDerivativeOrderAnnealing { get; set; } |
Gets or sets whether to use derivative-order annealing. |
bool UseFirstOrder { get; set; } |
Gets or sets whether to use first-order approximation. |
bool UseMultiStepLoss { get; set; } |
Gets or sets whether to use multi-step loss optimization (MSL). |
bool UsePerStepBatchNorm { get; set; } |
Gets or sets whether to use per-step batch normalization statistics. |
bool UsePerStepLearningRates { get; set; } |
Gets or sets whether to use per-step learning rates (LSLR). |
double CosineAnnealingMinRatio { get; set; } |
Gets or sets the cosine annealing schedule minimum learning rate ratio. |
double InnerLearningRate { get; set; } |
Gets or sets the base inner learning rate (used for initialization of per-step rates). |
double OuterLearningRate { get; set; } |
Gets or sets the outer learning rate for meta-parameter updates. |
double? GradientClipThreshold { get; set; } |
Gets or sets the gradient clipping threshold. |
double[] MultiStepLossWeights { get; set; } |
Gets or sets the weights for multi-step loss at each adaptation step. |
int AdaptationSteps { get; set; } |
Gets or sets the number of inner-loop adaptation steps. |
int AnnealingIterations { get; set; } |
Gets or sets the number of iterations over which to anneal from first-order to second-order. |
int CheckpointFrequency { get; set; } |
Gets or sets the checkpoint frequency. |
int EvaluationFrequency { get; set; } |
Gets or sets the evaluation frequency. |
int EvaluationTasks { get; set; } |
Gets or sets the number of evaluation tasks. |
int MetaBatchSize { get; set; } |
Gets or sets the number of tasks per meta-batch. |
int NumMetaIterations { get; set; } |
Gets or sets the total number of meta-training iterations. |
int? RandomSeed { get; set; } |
Gets or sets the random seed. |
Methods
| Method | Summary |
|---|---|
IMetaLearnerOptions<T> Clone() |
|
bool IsValid() |