FederatedEWC<T>

Models & Types · AiDotNet.FederatedLearning.ContinualLearning

Federated Elastic Weight Consolidation (EWC) — prevents forgetting by penalizing changes to parameters that are important for previously learned tasks.

Inheritance: FederatedLearningComponentBase<T> → FederatedEWC<T>

Implements: IFederatedContinualLearningStrategy<T>

For Beginners

Some model parameters are critical for recognizing cats, while others are critical for recognizing dogs. EWC identifies which parameters matter for which tasks and penalizes changing the important ones. In federated EWC, each client reports which parameters are important for their data, and the server keeps a global importance map.

How It Works

EWC (Kirkpatrick et al., 2017) uses the Fisher information matrix to estimate which parameters are important for each task. In federated EWC, the Fisher information is computed locally at each client and aggregated at the server to form a global estimate of parameter importance across all tasks and clients.

Reference: Kirkpatrick et al. (2017), “Overcoming Catastrophic Forgetting in Neural Networks”.

Constructors

Constructor Summary
FederatedEWC(int fisherSamples) Creates a new Federated EWC strategy.

Methods

Method Summary
T ComputeRegularizationPenalty(Vector<T> currentParameters, Vector<T> referenceParameters, Vector<T> importanceWeights, double regularizationStrength)
Vector<T> AggregateImportance(Dictionary<int, Vector<T>> clientImportances, Dictionary<int, double> clientWeights)
Vector<T> ComputeImportance(Vector<T> modelParameters, Matrix<T> taskData)
Vector<T> ProjectGradient(Vector<T> gradient, Vector<T> importanceWeights)