LoRAAdapterBase<T>
Base Classes · AiDotNet.LoRA.Adapters
Abstract base class for LoRA (Low-Rank Adaptation) adapters that wrap existing layers.
Inheritance: LayerBase<T> → LoRAAdapterBase<T>
Implements: ILoRAAdapter<T>, IShapeContract
Known derived types: AdaLoRAAdapter<T>, ChainLoRAAdapter<T>, DVoRAAdapter<T>, DeltaLoRAAdapter<T>, DenseLoRAAdapter<T>, DoRAAdapter<T>, DyLoRAAdapter<T>, FloraAdapter<T>, GLoRAAdapter<T>, GraphConvolutionalLoRAAdapter<T>, HRAAdapter<T>, LoHaAdapter<T>, LoKrAdapter<T>, LoRADropAdapter<T>, LoRAFAAdapter<T>, LoRAPlusAdapter<T>, LoRAXSAdapter<T>, LoRETTAAdapter<T>, LoftQAdapter<T>, LongLoRAAdapter<T>, MoRAAdapter<T>, MultiLoRAAdapter<T>, NOLAAdapter<T>, PiSSAAdapter<T>, QALoRAAdapter<T>, QLoRAAdapter<T>, ReLoRAAdapter<T>, RoSAAdapter<T>, SLoRAAdapter<T>, StandardLoRAAdapter<T>, TiedLoRAAdapter<T>, VBLoRAAdapter<T>, VeRAAdapter<T>, XLoRAAdapter<T>
For Beginners
This is the foundation for all LoRA adapters in the library.
A LoRA adapter wraps an existing layer (like a dense or convolutional layer) and adds a small “correction layer” that learns what adjustments are needed. This base class:
- Manages both the original layer and the LoRA correction layer
- Handles parameter synchronization between them
- Provides common forward/backward pass logic (original + correction)
- Lets specialized adapters handle layer-specific details
This design allows you to create LoRA adapters for any layer type by:
- Inheriting from this base class
- Implementing layer-specific validation
- Implementing how to merge the LoRA weights back into the original layer
The result is parameter-efficient fine-tuning that works across different layer architectures!
How It Works
This base class provides common functionality for all LoRA adapter implementations. It manages the base layer, LoRA layer, and parameter synchronization, while allowing derived classes to implement layer-type-specific logic such as merging and validation.
Properties
| Property | Summary |
|---|---|
ILayer<T> BaseLayer { get; } |
Gets the base layer being adapted with LoRA. |
LoRALayer<T> LoRALayer { get; } |
Gets the LoRA layer providing the low-rank adaptation. |
bool IsBaseLayerFrozen { get; } |
Gets whether the base layer’s parameters are frozen during training. |
bool SupportsTraining { get; } |
Gets whether this adapter supports training. |
double Alpha { get; } |
Gets the scaling factor (alpha) for the LoRA adaptation. |
int Rank { get; } |
Gets the rank of the low-rank decomposition. |
Methods
| Method | Summary |
|---|---|
ILayer<T> MergeToOriginalLayer() |
Merges the LoRA adaptation into the base layer and returns the merged layer. |
IReadOnlyList<ILayer<T>> GetSubLayers() |
|
IReadOnlyList<OutputAxisContract> OutputAxesFor(int inputRank) |
|
void ResetState() |
Resets the internal state of both the base layer and LoRA layer. |
void UpdateParameters(T learningRate) |
Updates parameters using the specified learning rate. |