AutoLoRAAlgorithm<T, TInput, TOutput>
Models & Types · AiDotNet.MetaLearning.Algorithms
Implementation of AutoLoRA: Automatically Tuning Matrix Ranks in Low-Rank Adaptation Based on Meta Learning (Zhang et al., NAACL 2024).
Inheritance: ModelBase<T, TInput, TOutput> → MetaLearnerBase<T, TInput, TOutput> → AutoLoRAAlgorithm<T, TInput, TOutput>
Implements: IGeneratedParameterRegistrar<T>
How It Works
AutoLoRA addresses LoRA’s limitation of uniform rank assignment across all layers by using a bi-level optimization to automatically discover the optimal rank for each parameter group.
Algorithm:
Constructors
| Constructor | Summary |
|---|---|
AutoLoRAAlgorithm(AutoLoRAOptions<T, TInput, TOutput> options) |
Properties
| Property | Summary |
|---|---|
MetaLearningAlgorithmType AlgorithmType { get; } |
Methods
| Method | Summary |
|---|---|
IModel<TInput, TOutput, ModelMetadata<T>> Adapt(IMetaLearningTask<T, TInput, TOutput> task) |
|
T MetaTrain(TaskBatch<T, TInput, TOutput> taskBatch) |