IFederatedAdapterStrategy<T>

Interfaces · AiDotNet.FederatedLearning.Adapters

Interface for federated adapter strategies that enable parameter-efficient fine-tuning (PEFT) in FL.

Known implementations: DPFedLoRA<T>, FLoRA<T>, FedMeZO<T>, FedPETuning<T>, FederatedAdapterTuning<T>, FederatedLoRA<T>, FederatedPromptTuning<T>, HeterogeneousLoRA<T>, HierarchicalFedLoRA<T>, SparseLoRA<T>

For Beginners

Large AI models (like GPT or LLaMA) have billions of parameters. Sending all these between clients and server is impractical. Adapters are tiny “add-on” modules (often <1% of total parameters) that customize the model. Only these adapters are shared in federated learning, making it practical to fine-tune massive models collaboratively.

How It Works

Federated adapter strategies allow clients to fine-tune foundation models by training only small adapter modules (LoRA, prompt tuning) rather than full model parameters. This dramatically reduces communication costs and enables FL for large language models.

Properties

Property Summary
double CompressionRatio { get; } Gets the compression ratio (adapter params / total model params).
int AdapterParameterCount { get; } Gets the total number of trainable adapter parameters.

Methods

Method Summary
Vector<T> AggregateAdapters(Dictionary<int, Vector<T>> clientAdapters, Dictionary<int, double> clientWeights) Aggregates adapter parameters from multiple clients.
Vector<T> ExtractAdapterParameters(Vector<T> fullModelParameters) Extracts adapter parameters from a full model parameter vector.
Vector<T> MergeAdapterParameters(Vector<T> fullModelParameters, Vector<T> aggregatedAdapters) Merges aggregated adapter parameters back into the full model.