AdaLoRAAdapter<T> |
Adaptive Low-Rank Adaptation (AdaLoRA) adapter that dynamically allocates parameter budgets among weight matrices. |
ChainLoRAAdapter<T> |
Chain-of-LoRA adapter that implements sequential composition of multiple LoRA adapters. |
DVoRAAdapter<T> |
DVoRA (DoRA + VeRA) adapter - combines DoRA’s magnitude-direction decomposition with VeRA’s extreme parameter efficiency. |
DeltaLoRAAdapter<T> |
Delta-LoRA adapter that focuses on parameter-efficient delta updates with momentum. |
DenseLoRAAdapter<T> |
LoRA adapter specifically for Dense and FullyConnected layers with 1D input/output shapes. |
DoRAAdapter<T> |
DoRA (Weight-Decomposed Low-Rank Adaptation) adapter for parameter-efficient fine-tuning with improved stability. |
DyLoRAAdapter<T> |
DyLoRA (Dynamic LoRA) adapter that trains with multiple ranks simultaneously. |
FloraAdapter<T> |
Implements Flora (Low-Rank Adapters Are Secretly Gradient Compressors) adapter for memory-efficient fine-tuning. |
GLoRAAdapter<T> |
Generalized LoRA (GLoRA) implementation that adapts both weights AND activations. |
GraphConvolutionalLoRAAdapter<T> |
LoRA adapter for Graph Convolutional layers, enabling parameter-efficient fine-tuning of GNN models. |
HRAAdapter<T> |
HRA (Hybrid Rank Adaptation) adapter that combines low-rank and full-rank updates for optimal parameter efficiency. |
LoHaAdapter<T> |
LoHa (Low-Rank Hadamard Product Adaptation) adapter for parameter-efficient fine-tuning. |
LoKrAdapter<T> |
LoKr (Low-Rank Kronecker Product Adaptation) adapter for parameter-efficient fine-tuning. |
LoRADropAdapter<T> |
LoRA-drop implementation: LoRA with dropout regularization. |
LoRAFAAdapter<T> |
LoRA-FA (LoRA with Frozen A matrix) adapter for parameter-efficient fine-tuning. |
LoRAPlusAdapter<T> |
LoRA+ adapter that uses optimized learning rates for faster convergence and better performance. |
LoRAXSAdapter<T> |
LoRA-XS (Extremely Small) adapter for ultra-parameter-efficient fine-tuning using SVD with trainable scaling matrix. |
LoRETTAAdapter<T> |
LoRETTA (Low-Rank Economic Tensor-Train Adaptation) adapter for parameter-efficient fine-tuning. |
LoftQAdapter<T> |
LoftQ (LoRA-Fine-Tuning-Quantized) adapter that combines quantization and LoRA with improved initialization. |
LongLoRAAdapter<T> |
LongLoRA adapter that efficiently extends LoRA to handle longer context lengths using shifted sparse attention. |
MoRAAdapter<T> |
Implements MoRA (High-Rank Updating for Parameter-Efficient Fine-Tuning) adapter. |
MultiLoRAAdapter<T> |
Multi-task LoRA adapter that manages multiple task-specific LoRA layers for complex multi-task learning scenarios. |
NOLAAdapter<T> |
Implements NOLA (Compressing LoRA using Linear Combination of Random Basis) adapter for extreme parameter efficiency. |
PiSSAAdapter<T> |
Principal Singular Values and Singular Vectors Adaptation (PiSSA) adapter for parameter-efficient fine-tuning. |
QALoRAAdapter<T> |
Quantization-Aware LoRA (QA-LoRA) adapter that combines parameter-efficient fine-tuning with group-wise quantization awareness. |
QLoRAAdapter<T> |
QLoRA (Quantized LoRA) adapter for parameter-efficient fine-tuning with 4-bit quantized base weights. |
ReLoRAAdapter<T> |
Restart LoRA (ReLoRA) adapter that periodically merges and restarts LoRA training for continual learning. |
RoSAAdapter<T> |
RoSA (Robust Adaptation) adapter for parameter-efficient fine-tuning with improved robustness to distribution shifts. |
SLoRAAdapter<T> |
S-LoRA adapter for scalable serving of thousands of concurrent LoRA adapters. |
StandardLoRAAdapter<T> |
Standard LoRA implementation (original LoRA algorithm). |
TiedLoRAAdapter<T> |
Tied-LoRA adapter - LoRA with weight tying for extreme parameter efficiency across deep networks. |
VBLoRAAdapter<T> |
Vector Bank LoRA (VB-LoRA) adapter that uses shared parameter banks for efficient multi-client deployment. |
VeRAAdapter<T> |
VeRA (Vector-based Random Matrix Adaptation) adapter - an extreme parameter-efficient variant of LoRA. |
XLoRAAdapter<T> |
X-LoRA (Mixture of LoRA Experts) adapter that uses multiple LoRA experts with learned routing. |