VBLoRAAdapter<T>
Models & Types · AiDotNet.LoRA.Adapters
Vector Bank LoRA (VB-LoRA) adapter that uses shared parameter banks for efficient multi-client deployment.
Inheritance: LayerBase<T> → LoRAAdapterBase<T> → VBLoRAAdapter<T>
For Beginners
Think of vector banks like a shared library of building blocks.
Traditional LoRA is like each person having their own complete toolbox:
- Person 1: Full set of tools
- Person 2: Full set of tools
- Person 3: Full set of tools
Result: Lots of duplicate tools
VB-LoRA is like a shared tool library:
- Central tool bank: One of each tool type
- Each person: List of which tools they need (indices)
- Everyone shares the same physical tools
Result: Much fewer tools needed overall
This is especially powerful when many adapters need similar adjustments (common in multi-task learning or personalization scenarios).
How It Works
VB-LoRA (2024) introduces vector banks - reusable parameter stores shared across multiple LoRA adapters. Instead of each adapter having its own complete A and B matrices, VB-LoRA maintains global banks of column vectors. Each adapter selects which vectors from the banks to use via index arrays.
Key Innovation: Vector Bank Architecture
Traditional LoRA:
- Each adapter stores full A (inputSize × rank) and B (rank × outputSize) matrices
- Total parameters for N adapters = N × (inputSize × rank + rank × outputSize)
- No sharing between adapters
VB-LoRA:
- Global BankA contains pooled column vectors (inputSize × bankSize)
- Global BankB contains pooled column vectors (bankSize × outputSize)
- Each adapter stores only indices (which vectors to use from banks)
- Total parameters = (inputSize × bankSize + bankSize × outputSize) + N × 2 × rank × sizeof(int)
- Massive reduction when bankSize << N × rank
Benefits:
- Reduced Duplication: Similar adapters share vector bank entries
- Lower Communication Overhead: Multi-client systems can cache banks locally
- Memory Efficiency: Fewer unique parameters to store and transmit
- Scalability: Adding new adapters only requires index arrays, not full matrices
- Knowledge Sharing: Banks capture common adaptation patterns
Example Scenario:
Suppose you’re deploying personalized language models to 1000 users:
- Traditional LoRA: Each user needs their own 16K parameter adapter (16MB total)
- VB-LoRA: Shared 256K parameter bank + 1000 users × 128 indices each
- Result: 84% memory reduction (256K + 128K vs 16M)
The shared bank captures common language patterns, while per-user indices select the patterns relevant to each individual.
Constructors
| Constructor | Summary |
|---|---|
VBLoRAAdapter(ILayer<T> baseLayer, int rank, int bankSizeA, int bankSizeB, int[] bankIndicesA, int[] bankIndicesB, double alpha, bool freezeBaseLayer, string bankKey) |
Initializes a new VB-LoRA adapter with specified bank sizes and indices. |
Properties
| Property | Summary |
|---|---|
int BankSizeA { get; } |
Gets the size of Bank A (number of available column vectors). |
int BankSizeB { get; } |
Gets the size of Bank B (number of available row vectors). |
int[] BankIndicesA { get; } |
Gets the indices into Bank A used by this adapter. |
int[] BankIndicesB { get; } |
Gets the indices into Bank B used by this adapter. |
Methods
| Method | Summary |
|---|---|
ILayer<T> MergeToOriginalLayer() |
Merges the VB-LoRA adaptation into the base layer and returns the merged layer. |
Matrix<T> GetBankA(string bankKey) |
Gets the global vector bank A for inspection or advanced use cases. |
Matrix<T> GetBankB(string bankKey) |
Gets the global vector bank B for inspection or advanced use cases. |
void ClearBanks(string bankKey) |
Clears the global vector banks (useful for testing or reinitialization). |
void UpdateParameters(T learningRate) |
Updates parameters and propagates changes back to the shared banks. |