IFederatedDistillationStrategy<T>
Interfaces · AiDotNet.FederatedLearning.Distillation
Interface for federated knowledge distillation strategies that enable model-heterogeneous FL.
Known implementations: FedDFDistillation<T>, FedGENDistillation<T>, FedMDDistillation<T>
For Beginners
Imagine different hospitals using different AI models (some simple, some complex). Knowledge distillation lets them share what they’ve learned without requiring everyone to use the same model. The “knowledge” is transferred through predictions or summaries rather than model weights.
How It Works
In standard FL, all clients must use the same model architecture. Federated knowledge distillation removes this constraint by exchanging knowledge (logits, prototypes, or generated samples) instead of raw model parameters.
Methods
| Method | Summary |
|---|---|
Matrix<T> AggregateKnowledge(Dictionary<int, Matrix<T>> clientKnowledge, Dictionary<int, double> clientWeights) |
Aggregates knowledge from multiple clients into a global knowledge representation. |
Matrix<T> ExtractKnowledge(Vector<T> localModelParameters, Matrix<T> publicData) |
Generates the knowledge representation to send to the server (logits, prototypes, etc.). |
Vector<T> ApplyKnowledge(Vector<T> localModelParameters, Matrix<T> globalKnowledge, Matrix<T> publicData, double temperature) |
Applies the aggregated global knowledge to update a local model. |