SLoRAAdapter<T>

Models & Types · AiDotNet.LoRA.Adapters

S-LoRA adapter for scalable serving of thousands of concurrent LoRA adapters.

Inheritance: LayerBase<T> → LoRAAdapterBase<T> → SLoRAAdapter<T>

For Beginners

S-LoRA solves a real-world problem in production AI systems.

The problem:

  • You have a large base model (like GPT or LLaMA)
  • You want to serve thousands of different LoRA adapters (one per customer, task, or use case)
  • Each adapter is small (few MB), but thousands of them won’t fit in GPU memory
  • Naive approaches either: load one adapter at a time (slow) or reserve memory for all (wasteful)

S-LoRA’s solution:

  • Unified memory pool: Dynamically manage adapter weights and cache together
  • Batched computation: Process multiple adapters in parallel efficiently
  • Adapter clustering: Group adapters by rank for optimized computation
  • On-demand loading: Fetch adapters from CPU to GPU memory only when needed

Key features implemented:

  1. Unified Memory Pool: Single pool for adapter weights (no pre-allocation waste)
  2. Adapter Clustering: Group adapters by rank for batched computation
  3. Dynamic Loading: Load adapters on-demand, evict when not needed
  4. Batched Forward Pass: Process multiple requests with different adapters simultaneously
  5. Memory Efficiency: Serve 100x more adapters than naive approaches

Research Paper Reference: “S-LoRA: Serving Thousands of Concurrent LoRA Adapters” Ying Sheng, Shiyi Cao, et al. (November 2023) arXiv:2311.03285

Performance (from paper):

  • Throughput: 4x improvement over vLLM, 30x over HuggingFace PEFT
  • Adapter capacity: 2,000+ concurrent adapters on single server
  • Memory efficiency: 75-90% GPU memory utilization
  • Scalability: Superlinear throughput scaling with more GPUs

Example usage:

// Create S-LoRA serving system for base layer
var baseLayer = new DenseLayer<double>(outputSize: 128);
var sloraAdapter = new SLoRAAdapter<double>(baseLayer, rank: 8);

// Register multiple adapters for different tasks
var adapter1 = new LoRALayer<double>(128, 128, rank: 8);
var adapter2 = new LoRALayer<double>(128, 128, rank: 8);
var adapter3 = new LoRALayer<double>(128, 128, rank: 4);
sloraAdapter.RegisterAdapter("customer_1", adapter1, rank: 8);
sloraAdapter.RegisterAdapter("customer_2", adapter2, rank: 8);
sloraAdapter.RegisterAdapter("task_classification", adapter3, rank: 4);

// Process batched requests efficiently — one adapter id per input
var inputs = new[] { Tensor<double>.CreateRandom(1, 128), Tensor<double>.CreateRandom(1, 128) };
var adapterIds = new[] { "customer_1", "task_classification" };
var outputs = sloraAdapter.BatchForward(inputs, adapterIds);

When to use S-LoRA:

  • Serving multiple LoRA adapters in production
  • Multi-tenant AI systems (one adapter per tenant)
  • Task-specific fine-tuning at scale
  • Limited GPU memory but many adapters
  • Need high throughput with many concurrent users

Differences from standard LoRA:

  • Standard LoRA: Single adapter, simple forward/backward pass
  • S-LoRA: Multiple adapters, optimized for concurrent serving, memory pooling

How It Works

S-LoRA (Scalable LoRA) is a system designed for efficient serving of many LoRA adapters simultaneously. Published in November 2023, it addresses the challenge of deploying thousands of task-specific LoRA adapters in production environments with limited GPU memory.

Example

using AiDotNet.LoRA;
using AiDotNet.LoRA.Adapters;

var adapter = new SLoRAAdapter<double>(null, rank: 8, alpha: 8, freezeBaseLayer: true);
var config = new DefaultLoRAConfiguration<double>(rank: 8, alpha: 8, loraAdapter: adapter);
Console.WriteLine($"Configured SLoRAAdapter (rank {config.Rank}).");

Constructors

Constructor Summary
SLoRAAdapter(ILayer<T> baseLayer, int rank, double alpha, int maxLoadedAdapters, bool freezeBaseLayer) Initializes a new S-LoRA adapter for scalable multi-adapter serving.

Properties

Property Summary
int LoadedAdapterCount { get; } Gets the number of adapters currently loaded in memory.
int MaxLoadedAdapters { get; } Gets the maximum number of adapters that can be loaded simultaneously.
int RankClusterCount { get; } Gets the number of rank clusters for batched computation optimization.
int TotalAdapterCount { get; } Gets the total number of registered adapters in the pool.

Methods

Method Summary
Dictionary<string, double> GetStatistics() Gets statistics about the current state of the S-LoRA system.
ILayer<T> MergeToOriginalLayer() Merges the primary adapter into the base layer and returns the merged layer.
List<string> GetRankCluster(int rank) Gets the list of adapter IDs in a specific rank cluster.
Tensor<T> Forward(Tensor<T> input, string adapterId) Performs batched forward pass with a specific adapter.
Tensor<T>[] BatchForward(Tensor<T>[] inputs, string[] adapterIds) Performs batched forward pass with multiple adapters simultaneously.
void ClearAdapters() Clears all adapters from the pool (useful for testing or reset).
void LoadAdapter(string adapterId) Loads an adapter from the pool into active memory (simulates GPU loading).
void RegisterAdapter(string adapterId, LoRALayer<T> loraLayer, int rank) Registers a new adapter in the unified memory pool.