FedDFDistillation<T>

Models & Types · AiDotNet.FederatedLearning.Distillation

FedDF — Federated ensemble distillation using model averaging on unlabeled public data.

Inheritance: FederatedLearningComponentBase<T> → FedDFDistillation<T>

Implements: IFederatedDistillationStrategy<T>

For Beginners

Think of this as a “committee vote.” Each client model votes on what the answer should be for some shared unlabeled examples. The server then trains a fresh model to match the committee’s averaged predictions. This means each client can use a completely different type of model.

How It Works

FedDF (Lin et al., 2020) performs federated distillation by treating each client model as an ensemble member. The server distills the ensemble’s collective predictions on unlabeled data into a single global model. Unlike FedMD, FedDF works with completely different model architectures and does not require labeled public data.

Reference: Lin et al. (2020), “Ensemble Distillation for Robust Model Fusion in Federated Learning”.

Constructors

Constructor Summary
FedDFDistillation(double temperature, int serverDistillationEpochs, double serverLearningRate) Creates a new FedDF distillation strategy.

Methods

Method Summary
Matrix<T> AggregateKnowledge(Dictionary<int, Matrix<T>> clientKnowledge, Dictionary<int, double> clientWeights)
Matrix<T> ExtractKnowledge(Vector<T> localModelParameters, Matrix<T> publicData)
Vector<T> ApplyKnowledge(Vector<T> localModelParameters, Matrix<T> globalKnowledge, Matrix<T> publicData, double temperature)