AutoLoRAAlgorithm<T, TInput, TOutput>

Models & Types · AiDotNet.MetaLearning.Algorithms

Implementation of AutoLoRA: Automatically Tuning Matrix Ranks in Low-Rank Adaptation Based on Meta Learning (Zhang et al., NAACL 2024).

Inheritance: ModelBase<T, TInput, TOutput> → MetaLearnerBase<T, TInput, TOutput> → AutoLoRAAlgorithm<T, TInput, TOutput>

Implements: IGeneratedParameterRegistrar<T>

How It Works

AutoLoRA addresses LoRA’s limitation of uniform rank assignment across all layers by using a bi-level optimization to automatically discover the optimal rank for each parameter group.

Algorithm:

Constructors

Constructor Summary
AutoLoRAAlgorithm(AutoLoRAOptions<T, TInput, TOutput> options)

Properties

Property Summary
MetaLearningAlgorithmType AlgorithmType { get; }

Methods

Method Summary
IModel<TInput, TOutput, ModelMetadata<T>> Adapt(IMetaLearningTask<T, TInput, TOutput> task)
T MetaTrain(TaskBatch<T, TInput, TOutput> taskBatch)