MAMLPlusPlusAlgorithm<T, TInput, TOutput>
Models & Types · AiDotNet.MetaLearning.Algorithms
Implementation of MAML++ (How to Train Your MAML) for few-shot learning.
Inheritance: ModelBase<T, TInput, TOutput> → MetaLearnerBase<T, TInput, TOutput> → MAMLPlusPlusAlgorithm<T, TInput, TOutput>
Implements: IGeneratedParameterRegistrar<T>
For Beginners
MAML++ is the “industrial strength” version of MAML:
Problems with vanilla MAML:
- Training is unstable (loss can explode randomly)
- One learning rate doesn’t work well for all adaptation steps
- Second-order gradients are noisy early in training
- Batch normalization statistics become stale during inner loop
MAML++ solutions:
- Multi-step loss: Check performance at EVERY step, not just the last
- Per-step learning rates: Each step gets its own rate, learned during training
- Derivative-order annealing: Start simple, gradually get more precise
- Per-step batch norm: Keep separate statistics for each step
Analogy: If MAML is a car, MAML++ adds:
- Anti-lock brakes (stability fixes)
- Cruise control (per-step learning rates)
- A GPS that updates gradually (derivative-order annealing)
- Multiple speedometers (per-step batch norm)
How It Works
MAML++ is a production-hardened MAML that addresses training instabilities through:
- Multi-Step Loss (MSL): Supervise every inner-loop step, not just the final one
- Per-Step Learning Rates (LSLR): Each adaptation step has its own learnable learning rate
- Derivative-Order Annealing: Gradually transition from first-order to second-order gradients
- Per-Step Batch Normalization: Separate BN statistics for each adaptation step
- Cosine Annealing: Learning rate schedule for the outer loop
Algorithm - MAML++:
Reference: Antoniou, A., Edwards, H., & Storkey, A. (2019). How to Train Your MAML. ICLR 2019.
Constructors
| Constructor | Summary |
|---|---|
MAMLPlusPlusAlgorithm(MAMLPlusPlusOptions<T, TInput, TOutput> options) |
Initializes a new instance of the MAML++ algorithm. |
Properties
| Property | Summary |
|---|---|
MetaLearningAlgorithmType AlgorithmType { get; } |
|
Vector<T> PerStepLearningRates { get; } |
Gets the current per-step learning rates. |
Methods
| Method | Summary |
|---|---|
IModel<TInput, TOutput, ModelMetadata<T>> Adapt(IMetaLearningTask<T, TInput, TOutput> task) |
Adapts the meta-learned model to a new task using MAML++ per-step learning rates. |
T MetaTrain(TaskBatch<T, TInput, TOutput> taskBatch) |
Performs one meta-training step with MAML++ enhancements. |