MetaLoRAAlgorithm<T, TInput, TOutput>

Models & Types · AiDotNet.MetaLearning.Algorithms

Implementation of Meta-LoRA: Low-Rank Adaptation for Meta-Learning (2024).

Inheritance: ModelBase<T, TInput, TOutput> → MetaLearnerBase<T, TInput, TOutput> → MetaLoRAAlgorithm<T, TInput, TOutput>

Implements: IGeneratedParameterRegistrar<T>

How It Works

Meta-LoRA applies Low-Rank Adaptation (LoRA) to the meta-learning inner loop. Instead of adapting all d model parameters per task (as in MAML), it meta-learns r low-rank basis vectors {v_1, …, v_r} in parameter space and only adapts r scalar coefficients {c_1, …, c_r} during the inner loop.

Algorithm:

Advantages over MAML: Inner loop only adapts r parameters instead of d, making adaptation much cheaper for large models. The low-rank constraint also acts as an implicit regularizer, preventing overfitting on few-shot support sets.

Constructors

Constructor Summary
MetaLoRAAlgorithm(MetaLoRAOptions<T, TInput, TOutput> options)

Properties

Property Summary
MetaLearningAlgorithmType AlgorithmType { get; }

Methods

Method Summary
IModel<TInput, TOutput, ModelMetadata<T>> Adapt(IMetaLearningTask<T, TInput, TOutput> task)
T MetaTrain(TaskBatch<T, TInput, TOutput> taskBatch)