FedMDDistillation<T>
Models & Types · AiDotNet.FederatedLearning.Distillation
FedMD — Model-agnostic federated learning via mutual distillation on a public dataset.
Inheritance: FederatedLearningComponentBase<T> → FedMDDistillation<T>
Implements: IFederatedDistillationStrategy<T>
For Beginners
All clients are given the same “test questions” (public dataset). Each client submits their answers (predicted probabilities). The server averages all answers to get a “consensus answer,” then each client learns from this consensus. This way, a simple model on a phone can learn from a powerful model on a server.
How It Works
FedMD (Li & Wang, 2019) enables heterogeneous model architectures across clients. Each client computes logits on a shared public dataset, the server averages the logits, and clients distill the consensus logits into their local models.
Reference: Li & Wang (2019), “FedMD: Heterogeneous Federated Learning via Model Distillation”.
Constructors
| Constructor | Summary |
|---|---|
FedMDDistillation(double temperature, double distillationAlpha, int distillationEpochs, double learningRate) |
Creates a new FedMD distillation strategy. |
Methods
| Method | Summary |
|---|---|
Matrix<T> AggregateKnowledge(Dictionary<int, Matrix<T>> clientKnowledge, Dictionary<int, double> clientWeights) |
|
Matrix<T> ExtractKnowledge(Vector<T> localModelParameters, Matrix<T> publicData) |
|
Vector<T> ApplyKnowledge(Vector<T> localModelParameters, Matrix<T> globalKnowledge, Matrix<T> publicData, double temperature) |