FedPETuning<T>
Models & Types · AiDotNet.FederatedLearning.Adapters
Implements FedPETuning — a unified framework for parameter-efficient fine-tuning (PEFT) in federated learning that supports multiple PEFT methods under one API.
Inheritance: FederatedLearningComponentBase<T> → FedPETuning<T>
Implements: IFederatedAdapterStrategy<T>
For Beginners
There are many ways to fine-tune a large model cheaply (LoRA, adapter layers, prefix tuning, BitFit, etc.). FedPETuning wraps them all into a single federated strategy so you can swap methods easily. It also applies federated-aware selection to decide which parameters each client should update, based on data heterogeneity.
How It Works
Supported methods:
Reference: Zhang, Z., et al. (2023). “Federated Learning for Parameter-Efficient Fine-Tuning of Foundation Models.” ACL 2023.
Constructors
| Constructor | Summary |
|---|---|
FedPETuning(int modelDim, PEFTMethod method, int bottleneckDim, int numAdaptedLayers, int layerDim) |
Creates a new FedPETuning strategy. |
Properties
| Property | Summary |
|---|---|
PEFTMethod Method { get; } |
Gets the PEFT method being used. |
double CompressionRatio { get; } |
|
int AdapterParameterCount { get; } |
|
int BottleneckDim { get; } |
Gets the bottleneck/rank dimension. |
Methods
| Method | Summary |
|---|---|
Dictionary<int, bool[]> SelectParametersByHeterogeneity(Dictionary<int, double[]> clientGradientNorms, double selectionRatio) |
Selects which adapter parameters each client should update based on data heterogeneity. |
Vector<T> AggregateAdapters(Dictionary<int, Vector<T>> clientAdapters, Dictionary<int, double> clientWeights) |
|
Vector<T> AggregateAdaptersMethodAware(Dictionary<int, Vector<T>> clientAdapters, Dictionary<int, double> clientWeights) |
Aggregates adapters for the configured PEFT method. |
Vector<T> ApplySelectionMask(Vector<T> adapters, bool[] mask) |
Applies a selection mask to adapter parameters, zeroing out unselected positions. |
Vector<T> ExtractAdapterParameters(Vector<T> fullModelParameters) |
|
Vector<T> MergeAdapterParameters(Vector<T> fullModelParameters, Vector<T> aggregatedAdapters) |