FedCILContinualLearning<T>

Models & Types · AiDotNet.FederatedLearning.ContinualLearning

Implements FedCIL — federated class-incremental learning with generative replay, model consolidation and consistency enforcement.

Inheritance: FederatedLearningComponentBase<T> → FedCILContinualLearning<T>

Implements: IFederatedContinualLearningStrategy<T>

For Beginners

Instead of storing old data, every participant keeps a small “imagination” model that can redraw examples of classes it has seen. When a new task arrives, clients practise on real new data and on imagined old data (so they do not forget), and the server merges the clients’ imagination models into one that knows every client’s classes.

How It Works

FedCIL (Qi et al., 2023) gives every client an auxiliary-classifier GAN (ACGAN): a conditional generator G(y, ε) and a discriminator whose shared hidden layer feeds a real/fake head and a class head (the task classifier). Clients train on their current task plus generative replay of earlier classes drawn from the global generator, and add consistency enforcement: λ·KL(p_global(x̃) ‖ p_local(x̃)) on the replayed samples x̃, which keeps the local classifier consistent with the global one on old classes. The server performs model consolidation: it averages the client ACGANs and then fine-tunes the average with ACGAN steps on synthetic data generated by each client generator for that client’s label set.

Network sizes come from the constructor (feature, noise and hidden dimensions), so the models stay small. The IFederatedContinualLearningStrategy members describe the classifier parameters only loosely: FedCIL prevents forgetting through replay, so importance is uniform and gradients pass through.

Reference: Qi, D., Zhao, H., Li, S. (2023). “Better Generative Replay for Continual Federated Learning.” ICLR 2023.

Constructors

Constructor Summary
FedCILContinualLearning(int numClasses, int featureDim, int noiseDim, int hiddenDim, int batchSize, double learningRate, double consistencyWeight, int consolidationSteps, int seed) Creates a new FedCIL strategy.

Properties

Property Summary
IReadOnlyCollection<int> KnownClasses { get; } Gets the classes the server has consolidated so far.
double ConsistencyWeight { get; } Gets the consistency weight λ.
int ParameterCount { get; } Number of parameters of one ACGAN, in the order of GetParameters: generator W1 [(C + noise), H], b1 [H], W2 [H, D], b2 [D]; discriminator Wh [D, H], bh [H], Wadv [H, 1], badv [1], Wcls [H, C], bcls [C].

Methods

Method Summary
FedCILGenerativeModel<T> Consolidate(Dictionary<int, FedCILGenerativeModel<T>> clientModels, Dictionary<int, IReadOnlyCollection<int>> clientClasses, Dictionary<int, double> clientWeights) Server model consolidation: weighted average of the client ACGANs, then ACGAN fine-tuning on synthetic data that each client generator produces for its own label set.
FedCILGenerativeModel<T> CreateModel() Creates a randomly initialized ACGAN (Xavier-uniform weights, zero biases).
FedCILGenerativeModel<T> CreateModel(Vector<T> parameters) Creates an ACGAN from a flat parameter vector laid out as described by ParameterCount.
FedCILGenerativeModel<T> TrainClient(FedCILGenerativeModel<T> globalModel, Matrix<T> features, int[] labels, IReadOnlyCollection<int> previousClasses, int steps) Client update for one task: ACGAN training on the task data plus generative replay of previousClasses from the global generator, with consistency enforcement against the global classifier on the replayed samples.
Matrix<T> GenerateReplay(FedCILGenerativeModel<T> model, int classLabel, int numSamples) Draws replay samples of one class from a model’s generator.
T ComputeConsistencyLoss(FedCILGenerativeModel<T> globalModel, FedCILGenerativeModel<T> localModel, Matrix<T> samples) Consistency loss: mean over rows of KL(p_global(x) ‖ p_local(x)) between the classifier heads.
T ComputeRegularizationPenalty(Vector<T> currentParameters, Vector<T> referenceParameters, Vector<T> importanceWeights, double regularizationStrength)
Vector<T> AggregateImportance(Dictionary<int, Vector<T>> clientImportances, Dictionary<int, double> clientWeights)
Vector<T> ComputeImportance(Vector<T> modelParameters, Matrix<T> taskData)
Vector<T> GetParameters(FedCILGenerativeModel<T> model) Flattens an ACGAN into the parameter layout described by ParameterCount.
Vector<T> ProjectGradient(Vector<T> gradient, Vector<T> importanceWeights)
int[] Classify(FedCILGenerativeModel<T> model, Matrix<T> features) Predicts the class of every row with the model’s classifier head.