FedPETuning<T>

Models & Types · AiDotNet.FederatedLearning.Adapters

Implements FedPETuning — a unified framework for parameter-efficient fine-tuning (PEFT) in federated learning that supports multiple PEFT methods under one API.

Inheritance: FederatedLearningComponentBase<T> → FedPETuning<T>

Implements: IFederatedAdapterStrategy<T>

For Beginners

There are many ways to fine-tune a large model cheaply (LoRA, adapter layers, prefix tuning, BitFit, etc.). FedPETuning wraps them all into a single federated strategy so you can swap methods easily. It also applies federated-aware selection to decide which parameters each client should update, based on data heterogeneity.

How It Works

Supported methods:

Reference: Zhang, Z., et al. (2023). “Federated Learning for Parameter-Efficient Fine-Tuning of Foundation Models.” ACL 2023.

Constructors

Constructor Summary
FedPETuning(int modelDim, PEFTMethod method, int bottleneckDim, int numAdaptedLayers, int layerDim) Creates a new FedPETuning strategy.

Properties

Property Summary
PEFTMethod Method { get; } Gets the PEFT method being used.
double CompressionRatio { get; }
int AdapterParameterCount { get; }
int BottleneckDim { get; } Gets the bottleneck/rank dimension.

Methods

Method Summary
Dictionary<int, bool[]> SelectParametersByHeterogeneity(Dictionary<int, double[]> clientGradientNorms, double selectionRatio) Selects which adapter parameters each client should update based on data heterogeneity.
Vector<T> AggregateAdapters(Dictionary<int, Vector<T>> clientAdapters, Dictionary<int, double> clientWeights)
Vector<T> AggregateAdaptersMethodAware(Dictionary<int, Vector<T>> clientAdapters, Dictionary<int, double> clientWeights) Aggregates adapters for the configured PEFT method.
Vector<T> ApplySelectionMask(Vector<T> adapters, bool[] mask) Applies a selection mask to adapter parameters, zeroing out unselected positions.
Vector<T> ExtractAdapterParameters(Vector<T> fullModelParameters)
Vector<T> MergeAdapterParameters(Vector<T> fullModelParameters, Vector<T> aggregatedAdapters)