ILoRAAdapter<T>

Interfaces · AiDotNet.Interfaces

Interface for LoRA (Low-Rank Adaptation) adapters that wrap existing layers with parameter-efficient adaptations.

Implements: ILayer<T>

Known implementations: LoRAAdapterBase<T>

For Beginners

A LoRA adapter wraps an existing layer (like a dense or convolutional layer) and adds a small “correction layer” that learns what adjustments are needed. This is much more memory-efficient than retraining all the weights in a large model.

Think of it like:

  • The base layer has the original knowledge (frozen or trainable)
  • The LoRA layer learns a small correction
  • The final output combines both: original + correction

This allows you to adapt large pre-trained models with 100x fewer trainable parameters!

How It Works

LoRA adapters enable efficient fine-tuning of neural networks by learning low-rank decompositions of weight updates instead of modifying all weights directly. This interface defines the contract for all LoRA adapter implementations across different layer types.

Properties

Property Summary
ILayer<T> BaseLayer { get; } Gets the base layer being adapted with LoRA.
LoRALayer<T> LoRALayer { get; } Gets the LoRA layer providing the low-rank adaptation.
bool IsBaseLayerFrozen { get; } Gets whether the base layer’s parameters are frozen during training.
double Alpha { get; } Gets the scaling factor (alpha) for the LoRA adaptation.
int Rank { get; } Gets the rank of the low-rank decomposition.

Methods

Method Summary
ILayer<T> MergeToOriginalLayer() Merges the LoRA weights back into the original layer for deployment.