R2D2Options<T, TInput, TOutput>
Options & Configuration · AiDotNet.MetaLearning.Options
Configuration options for R2-D2 (Meta-learning with Differentiable Closed-form Solvers) algorithm.
Inheritance: ModelOptions → R2D2Options<T, TInput, TOutput>
Implements: IMetaLearnerOptions<T>
For Beginners
R2-D2 is like MAML but with a mathematical shortcut:
The Problem with MAML’s inner loop:
- MAML takes K gradient steps to adapt (iterative, slow)
- Each step requires computing gradients (expensive)
- More steps = more memory for second-order gradients
R2-D2’s solution:
- Instead of gradient steps, use ridge regression (closed-form, instant)
- Ridge regression has an exact mathematical solution: w = (X^T X + lambda I)^-1 X^T y
- This solution is differentiable, so meta-gradients still flow through it
- Result: Lightning-fast inner loop with one matrix solve instead of K gradient steps
Analogy:
- MAML: Walk toward the solution step by step (might need 5-10 steps)
- R2-D2: Jump directly to the solution using math (one step, exact answer)
The catch: R2-D2’s “jump” only works for the last layer (linear classifier), so the feature extractor still needs good meta-learned features.
How It Works
R2-D2 replaces the iterative inner-loop optimization of MAML with a closed-form differentiable ridge regression solver. This makes the inner loop extremely fast (single forward pass) while remaining fully differentiable for meta-gradient computation.
Reference: Bertinetto, L., Henriques, J. F., Torr, P., & Vedaldi, A. (2019). Meta-learning with Differentiable Closed-form Solvers. ICLR 2019.
Constructors
| Constructor | Summary |
|---|---|
R2D2Options(IFullModel<T, TInput, TOutput> metaModel) |
Initializes a new instance of the R2D2Options class. |
Properties
| Property | Summary |
|---|---|
IEpisodicDataLoader<T, TInput, TOutput> DataLoader { get; set; } |
Gets or sets the episodic data loader. |
IFullModel<T, TInput, TOutput> MetaModel { get; set; } |
Gets or sets the feature extractor model (backbone). |
IGradientBasedOptimizer<T, TInput, TOutput> InnerOptimizer { get; set; } |
Must stay null: this algorithm takes no plain gradient inner step (it adapts without one, or by its own update rule), so a supplied optimizer is rejected when the learner is built. |
IGradientBasedOptimizer<T, TInput, TOutput> MetaOptimizer { get; set; } |
Optimizer for the meta-update. |
ILossFunction<T> LossFunction { get; set; } |
Loss function for meta-training; null uses the default multi-class classification loss. |
bool EnableCheckpointing { get; set; } |
Gets or sets whether to save checkpoints. |
bool LearnLambda { get; set; } |
Gets or sets whether lambda should be meta-learned. |
bool UseFirstOrder { get; set; } |
Gets or sets whether to use first-order approximation. |
bool UseWoodburyIdentity { get; set; } |
Gets or sets whether to use Woodbury identity for efficient matrix inversion. |
double InnerLearningRate { get; set; } |
Gets or sets the inner learning rate (not used directly; R2-D2 uses closed-form solver). |
double Lambda { get; set; } |
Gets or sets the ridge regression regularization parameter (lambda). |
double OuterLearningRate { get; set; } |
Gets or sets the outer learning rate. |
double? GradientClipThreshold { get; set; } |
Gets or sets the gradient clipping threshold. |
int AdaptationSteps { get; set; } |
Gets or sets the number of adaptation steps (set to 1 for R2-D2’s closed-form solver). |
int CheckpointFrequency { get; set; } |
Gets or sets checkpoint frequency. |
int EmbeddingDimension { get; set; } |
Gets or sets the feature embedding dimension for ridge regression. |
int EvaluationFrequency { get; set; } |
Gets or sets evaluation frequency. |
int EvaluationTasks { get; set; } |
Gets or sets evaluation tasks count. |
int MetaBatchSize { get; set; } |
Gets or sets the meta-batch size. |
int NumMetaIterations { get; set; } |
Gets or sets the total number of meta-training iterations. |
int? RandomSeed { get; set; } |
Gets or sets the random seed. |
Methods
| Method | Summary |
|---|---|
IMetaLearnerOptions<T> Clone() |
|
bool IsValid() |