IFederatedAdapterStrategy<T>
Interfaces · AiDotNet.FederatedLearning.Adapters
Interface for federated adapter strategies that enable parameter-efficient fine-tuning (PEFT) in FL.
Known implementations: DPFedLoRA<T>, FLoRA<T>, FedMeZO<T>, FedPETuning<T>, FederatedAdapterTuning<T>, FederatedLoRA<T>, FederatedPromptTuning<T>, HeterogeneousLoRA<T>, HierarchicalFedLoRA<T>, SparseLoRA<T>
For Beginners
Large AI models (like GPT or LLaMA) have billions of parameters. Sending all these between clients and server is impractical. Adapters are tiny “add-on” modules (often <1% of total parameters) that customize the model. Only these adapters are shared in federated learning, making it practical to fine-tune massive models collaboratively.
How It Works
Federated adapter strategies allow clients to fine-tune foundation models by training only small adapter modules (LoRA, prompt tuning) rather than full model parameters. This dramatically reduces communication costs and enables FL for large language models.
Properties
| Property | Summary |
|---|---|
double CompressionRatio { get; } |
Gets the compression ratio (adapter params / total model params). |
int AdapterParameterCount { get; } |
Gets the total number of trainable adapter parameters. |
Methods
| Method | Summary |
|---|---|
Vector<T> AggregateAdapters(Dictionary<int, Vector<T>> clientAdapters, Dictionary<int, double> clientWeights) |
Aggregates adapter parameters from multiple clients. |
Vector<T> ExtractAdapterParameters(Vector<T> fullModelParameters) |
Extracts adapter parameters from a full model parameter vector. |
Vector<T> MergeAdapterParameters(Vector<T> fullModelParameters, Vector<T> aggregatedAdapters) |
Merges aggregated adapter parameters back into the full model. |