MetaLoRAAlgorithm<T, TInput, TOutput>
Models & Types · AiDotNet.MetaLearning.Algorithms
Implementation of Meta-LoRA: Low-Rank Adaptation for Meta-Learning (2024).
Inheritance: ModelBase<T, TInput, TOutput> → MetaLearnerBase<T, TInput, TOutput> → MetaLoRAAlgorithm<T, TInput, TOutput>
Implements: IGeneratedParameterRegistrar<T>
How It Works
Meta-LoRA applies Low-Rank Adaptation (LoRA) to the meta-learning inner loop. Instead of adapting all d model parameters per task (as in MAML), it meta-learns r low-rank basis vectors {v_1, …, v_r} in parameter space and only adapts r scalar coefficients {c_1, …, c_r} during the inner loop.
Algorithm:
Advantages over MAML: Inner loop only adapts r parameters instead of d, making adaptation much cheaper for large models. The low-rank constraint also acts as an implicit regularizer, preventing overfitting on few-shot support sets.
Constructors
| Constructor | Summary |
|---|---|
MetaLoRAAlgorithm(MetaLoRAOptions<T, TInput, TOutput> options) |
Properties
| Property | Summary |
|---|---|
MetaLearningAlgorithmType AlgorithmType { get; } |
Methods
| Method | Summary |
|---|---|
IModel<TInput, TOutput, ModelMetadata<T>> Adapt(IMetaLearningTask<T, TInput, TOutput> task) |
|
T MetaTrain(TaskBatch<T, TInput, TOutput> taskBatch) |