iMAMLAlgorithm<T, TInput, TOutput>
Models & Types · AiDotNet.MetaLearning.Algorithms
Implementation of the iMAML (Implicit Model-Agnostic Meta-Learning) algorithm.
Inheritance: ModelBase<T, TInput, TOutput> → MetaLearnerBase<T, TInput, TOutput> → iMAMLAlgorithm<T, TInput, TOutput>
For Beginners
iMAML solves one of MAML’s biggest problems - memory usage.
How It Works
iMAML is a memory-efficient variant of MAML that uses implicit differentiation to compute meta-gradients. Instead of backpropagating through all adaptation steps, it uses the implicit function theorem to directly compute gradients at the adapted parameters, significantly reducing memory requirements.
Key advantages over MAML:
- Constant memory cost regardless of number of adaptation steps
- Can use many more adaptation steps without memory issues
- Often achieves better performance than first-order MAML (FOMAML)
The problem with MAML:
- To learn from adaptation, MAML needs to remember every step
- More adaptation steps = much more memory needed
- This limits how much adaptation you can do
How iMAML solves it:
- Uses a mathematical shortcut (implicit differentiation)
- Only needs to remember the start and end points
- Can do many more adaptation steps with the same memory
The implicit function theorem allows computing gradients through the adaptation process by solving: (I + H / lambda)^(-1) * g, where H is the Hessian of the inner support loss, lambda the strength of the proximal regularizer each inner problem carries, and g is the gradient of the query loss. This is solved efficiently using Conjugate Gradient iteration.
Reference: Rajeswaran, A., Finn, C., Kakade, S. M., & Levine, S. (2019). Meta-learning with implicit gradients.
Constructors
| Constructor | Summary |
|---|---|
iMAMLAlgorithm(iMAMLOptions<T, TInput, TOutput> options) |
Initializes a new instance of the iMAMLAlgorithm class. |
Properties
| Property | Summary |
|---|---|
MetaLearningAlgorithmType AlgorithmType { get; } |
Gets the algorithm type identifier for this meta-learner. |
Methods
| Method | Summary |
|---|---|
IModel<TInput, TOutput, ModelMetadata<T>> Adapt(IMetaLearningTask<T, TInput, TOutput> task) |
Adapts the meta-learned model to a new task using iMAML’s inner loop optimization. |
T MetaTrain(TaskBatch<T, TInput, TOutput> taskBatch) |
Performs one meta-training step using iMAML’s implicit gradient computation. |