FederatedEWC<T>
Models & Types · AiDotNet.FederatedLearning.ContinualLearning
Federated Elastic Weight Consolidation (EWC) — prevents forgetting by penalizing changes to parameters that are important for previously learned tasks.
Inheritance: FederatedLearningComponentBase<T> → FederatedEWC<T>
Implements: IFederatedContinualLearningStrategy<T>
For Beginners
Some model parameters are critical for recognizing cats, while others are critical for recognizing dogs. EWC identifies which parameters matter for which tasks and penalizes changing the important ones. In federated EWC, each client reports which parameters are important for their data, and the server keeps a global importance map.
How It Works
EWC (Kirkpatrick et al., 2017) uses the Fisher information matrix to estimate which parameters are important for each task. In federated EWC, the Fisher information is computed locally at each client and aggregated at the server to form a global estimate of parameter importance across all tasks and clients.
Reference: Kirkpatrick et al. (2017), “Overcoming Catastrophic Forgetting in Neural Networks”.
Constructors
| Constructor | Summary |
|---|---|
FederatedEWC(int fisherSamples) |
Creates a new Federated EWC strategy. |
Methods
| Method | Summary |
|---|---|
T ComputeRegularizationPenalty(Vector<T> currentParameters, Vector<T> referenceParameters, Vector<T> importanceWeights, double regularizationStrength) |
|
Vector<T> AggregateImportance(Dictionary<int, Vector<T>> clientImportances, Dictionary<int, double> clientWeights) |
|
Vector<T> ComputeImportance(Vector<T> modelParameters, Matrix<T> taskData) |
|
Vector<T> ProjectGradient(Vector<T> gradient, Vector<T> importanceWeights) |