DPFedLoRA<T>

Models & Types · AiDotNet.FederatedLearning.Adapters

Implements DP-FedLoRA — Differentially Private Federated LoRA aggregation.

Inheritance: FederatedLearningComponentBase<T> → DPFedLoRA<T>

Implements: IFederatedAdapterStrategy<T>

For Beginners

Each client’s LoRA adapter update is clipped to a fixed L2 norm, the clipped updates are averaged, and Gaussian noise calibrated to that clip norm is added, so no single client’s data can be inferred from the aggregate.

How It Works

Gaussian mechanism (DP-FedAvg, McMahan et al. 2018):

The noise scale is data independent. Deriving per-layer noise from the clients’ own adapter norms would make the noise depend on private data and under-noise the release.

Reference: DP-FedLoRA: Differentially Private Federated LoRA Fine-Tuning (2024).

Constructors

Constructor Summary
DPFedLoRA(int modelDim, int rank, double alpha, double noiseMultiplier, double clipNorm, int numAdaptedLayers, int layerInputDim, int layerOutputDim, int seed) Creates a new DP-FedLoRA strategy.

Properties

Property Summary
double ClipNorm { get; } Gets the per-sample clip norm.
double CompressionRatio { get; }
double CumulativeRdpEpsilon { get; } Gets the current cumulative RDP epsilon (before conversion to (eps,delta)-DP).
double LoraScaling { get; } Gets the LoRA output scaling alpha / rank that the forward pass applies to B*A.
double NoiseMultiplier { get; } Gets the DP noise multiplier.
int AdapterParameterCount { get; }
int Rank { get; } Gets the LoRA rank.
int RoundsCompleted { get; } Gets the number of aggregation rounds completed so far.

Methods

Method Summary
Vector<T> AggregateAdapters(Dictionary<int, Vector<T>> clientAdapters, Dictionary<int, double> clientWeights)
Vector<T> ExtractAdapterParameters(Vector<T> fullModelParameters)
Vector<T> MergeAdapterParameters(Vector<T> fullModelParameters, Vector<T> aggregatedAdapters)
double ComputePrivacySpent(double delta) Computes the cumulative (epsilon, delta)-DP guarantee after all rounds so far.
int EstimateMaxRounds(double targetEpsilon, double delta, int numClientsPerRound) Estimates the maximum number of rounds that can be performed while staying within a target epsilon budget.