FedDFDistillation<T>
Models & Types · AiDotNet.FederatedLearning.Distillation
FedDF — Federated ensemble distillation using model averaging on unlabeled public data.
Inheritance: FederatedLearningComponentBase<T> → FedDFDistillation<T>
Implements: IFederatedDistillationStrategy<T>
For Beginners
Think of this as a “committee vote.” Each client model votes on what the answer should be for some shared unlabeled examples. The server then trains a fresh model to match the committee’s averaged predictions. This means each client can use a completely different type of model.
How It Works
FedDF (Lin et al., 2020) performs federated distillation by treating each client model as an ensemble member. The server distills the ensemble’s collective predictions on unlabeled data into a single global model. Unlike FedMD, FedDF works with completely different model architectures and does not require labeled public data.
Reference: Lin et al. (2020), “Ensemble Distillation for Robust Model Fusion in Federated Learning”.
Constructors
| Constructor | Summary |
|---|---|
FedDFDistillation(double temperature, int serverDistillationEpochs, double serverLearningRate) |
Creates a new FedDF distillation strategy. |
Methods
| Method | Summary |
|---|---|
Matrix<T> AggregateKnowledge(Dictionary<int, Matrix<T>> clientKnowledge, Dictionary<int, double> clientWeights) |
|
Matrix<T> ExtractKnowledge(Vector<T> localModelParameters, Matrix<T> publicData) |
|
Vector<T> ApplyKnowledge(Vector<T> localModelParameters, Matrix<T> globalKnowledge, Matrix<T> publicData, double temperature) |