R2D2Algorithm<T, TInput, TOutput>
Models & Types · AiDotNet.MetaLearning.Algorithms
Implementation of R2-D2 (Meta-learning with Differentiable Closed-form Solvers) for few-shot learning.
Inheritance: ModelBase<T, TInput, TOutput> → MetaLearnerBase<T, TInput, TOutput> → R2D2Algorithm<T, TInput, TOutput>
For Beginners
R2-D2 is a clever approach to few-shot learning:
The Key Insight: Instead of slowly learning a classifier through gradient descent (like MAML does), R2-D2 computes the optimal classifier instantly using a mathematical formula.
How it works:
- Pass support set through the feature extractor to get features
- Solve ridge regression on those features: w = (X^T X + lambda I)^-1 X^T y
- This gives the OPTIMAL linear classifier in one step!
- Evaluate on query set using this optimal classifier
- Backpropagate through the entire process (including the matrix solve)
- Update the feature extractor to produce better features
Why it works:
- The ridge regression formula has a known, exact derivative
- We can backpropagate through matrix inversion (using implicit differentiation)
- This trains the feature extractor to produce features that are easy to classify
Analogy: Traditional few-shot (MAML): “Here are 5 photos. Let me practice classifying them for 10 rounds… okay, now I’m ready.” R2-D2: “Here are 5 photos. does instant math I know the optimal classifier. Done.”
How It Works
R2-D2 replaces MAML’s iterative inner-loop gradient descent with a differentiable closed-form ridge regression solver. The feature extractor (backbone) is meta-learned, and the final classifier is computed analytically using ridge regression on the extracted features.
Algorithm - R2-D2:
Key Insights:
- Closed-Form = No Inner Loop: Ridge regression has an exact solution, so there’s
no iterative optimization. This is both faster and more stable than MAML.
- Differentiable Matrix Solve: The gradient of the loss with respect to the features
flows through the matrix inversion using the identity: d(A^-1)/dA = -A^-1 (dA) A^-1.
- Feature Learning: The backbone learns to produce features where ridge regression
works well, which means linearly separable features with good margins.
- Woodbury Identity: When n_support << d (few-shot), we can use the Woodbury
identity to invert a smaller n x n matrix instead of d x d.
Reference: Bertinetto, L., Henriques, J. F., Torr, P., & Vedaldi, A. (2019). Meta-learning with Differentiable Closed-form Solvers. ICLR 2019.
Constructors
| Constructor | Summary |
|---|---|
R2D2Algorithm(R2D2Options<T, TInput, TOutput> options) |
Initializes a new instance of the R2-D2 algorithm. |
Properties
| Property | Summary |
|---|---|
MetaLearningAlgorithmType AlgorithmType { get; } |
|
double Lambda { get; } |
Gets the current lambda (regularization) value. |
Methods
| Method | Summary |
|---|---|
IModel<TInput, TOutput, ModelMetadata<T>> Adapt(IMetaLearningTask<T, TInput, TOutput> task) |
Adapts to a new task by computing the ridge regression classifier on support features. |
T MetaTrain(TaskBatch<T, TInput, TOutput> taskBatch) |
Performs one meta-training step using differentiable ridge regression. |