R2D2Options<T, TInput, TOutput>

Options & Configuration · AiDotNet.MetaLearning.Options

Configuration options for R2-D2 (Meta-learning with Differentiable Closed-form Solvers) algorithm.

Inheritance: ModelOptions → R2D2Options<T, TInput, TOutput>

Implements: IMetaLearnerOptions<T>

For Beginners

R2-D2 is like MAML but with a mathematical shortcut:

The Problem with MAML’s inner loop:

  • MAML takes K gradient steps to adapt (iterative, slow)
  • Each step requires computing gradients (expensive)
  • More steps = more memory for second-order gradients

R2-D2’s solution:

  • Instead of gradient steps, use ridge regression (closed-form, instant)
  • Ridge regression has an exact mathematical solution: w = (X^T X + lambda I)^-1 X^T y
  • This solution is differentiable, so meta-gradients still flow through it
  • Result: Lightning-fast inner loop with one matrix solve instead of K gradient steps

Analogy:

  • MAML: Walk toward the solution step by step (might need 5-10 steps)
  • R2-D2: Jump directly to the solution using math (one step, exact answer)

The catch: R2-D2’s “jump” only works for the last layer (linear classifier), so the feature extractor still needs good meta-learned features.

How It Works

R2-D2 replaces the iterative inner-loop optimization of MAML with a closed-form differentiable ridge regression solver. This makes the inner loop extremely fast (single forward pass) while remaining fully differentiable for meta-gradient computation.

Reference: Bertinetto, L., Henriques, J. F., Torr, P., & Vedaldi, A. (2019). Meta-learning with Differentiable Closed-form Solvers. ICLR 2019.

Constructors

Constructor Summary
R2D2Options(IFullModel<T, TInput, TOutput> metaModel) Initializes a new instance of the R2D2Options class.

Properties

Property Summary
IEpisodicDataLoader<T, TInput, TOutput> DataLoader { get; set; } Gets or sets the episodic data loader.
IFullModel<T, TInput, TOutput> MetaModel { get; set; } Gets or sets the feature extractor model (backbone).
IGradientBasedOptimizer<T, TInput, TOutput> InnerOptimizer { get; set; } Must stay null: this algorithm takes no plain gradient inner step (it adapts without one, or by its own update rule), so a supplied optimizer is rejected when the learner is built.
IGradientBasedOptimizer<T, TInput, TOutput> MetaOptimizer { get; set; } Optimizer for the meta-update.
ILossFunction<T> LossFunction { get; set; } Loss function for meta-training; null uses the default multi-class classification loss.
bool EnableCheckpointing { get; set; } Gets or sets whether to save checkpoints.
bool LearnLambda { get; set; } Gets or sets whether lambda should be meta-learned.
bool UseFirstOrder { get; set; } Gets or sets whether to use first-order approximation.
bool UseWoodburyIdentity { get; set; } Gets or sets whether to use Woodbury identity for efficient matrix inversion.
double InnerLearningRate { get; set; } Gets or sets the inner learning rate (not used directly; R2-D2 uses closed-form solver).
double Lambda { get; set; } Gets or sets the ridge regression regularization parameter (lambda).
double OuterLearningRate { get; set; } Gets or sets the outer learning rate.
double? GradientClipThreshold { get; set; } Gets or sets the gradient clipping threshold.
int AdaptationSteps { get; set; } Gets or sets the number of adaptation steps (set to 1 for R2-D2’s closed-form solver).
int CheckpointFrequency { get; set; } Gets or sets checkpoint frequency.
int EmbeddingDimension { get; set; } Gets or sets the feature embedding dimension for ridge regression.
int EvaluationFrequency { get; set; } Gets or sets evaluation frequency.
int EvaluationTasks { get; set; } Gets or sets evaluation tasks count.
int MetaBatchSize { get; set; } Gets or sets the meta-batch size.
int NumMetaIterations { get; set; } Gets or sets the total number of meta-training iterations.
int? RandomSeed { get; set; } Gets or sets the random seed.

Methods

Method Summary
IMetaLearnerOptions<T> Clone()
bool IsValid()