MoonAggregationStrategy<T>

Models & Types · AiDotNet.FederatedLearning.Aggregators

Implements MOON (Model-COntrastive Learning) aggregation strategy.

Inheritance: FederatedLearningComponentBase<T> → AggregationStrategyBase<Dictionary<string, T[]>, T> → ParameterDictionaryAggregationStrategyBase<T> → MoonAggregationStrategy<T>

For Beginners

MOON corrects “local drift” by adding a contrastive loss during client training. The contrastive loss pulls the local model’s representation closer to the global model and pushes it away from the previous local model, reducing divergence caused by non-IID data.

How It Works

During aggregation, MOON uses standard weighted averaging (same as FedAvg). The key innovation is in the local training objective, which includes:

The contrastive loss uses cosine similarity scaled by temperature:

Reference: Li, Q., He, B., and Song, D. (2021). “Model-Contrastive Federated Learning.” CVPR 2021.

Constructors

Constructor Summary
MoonAggregationStrategy(double contrastiveWeight, double temperature) Initializes a new instance of the MoonAggregationStrategy class.

Properties

Property Summary
double ContrastiveWeight { get; } Gets the contrastive loss weight (mu).
double Temperature { get; } Gets the contrastive temperature parameter.

Methods

Method Summary
Dictionary<string, T[]> Aggregate(Dictionary<int, Dictionary<string, T[]>> clientModels, Dictionary<int, double> clientWeights) Aggregates client models using weighted averaging.
T ComputeContrastiveLoss(Vector<T> localRepresentation, Vector<T> globalRepresentation, Vector<T> previousLocalRepresentation) Computes the MOON contrastive loss for a client during local training.
T ComputeTotalLoss(T taskLoss, Vector<T> localRepresentation, Vector<T> globalRepresentation, int clientId) Computes the complete local training loss including both task loss and contrastive loss.
Vector<T> GetPreviousRepresentation(int clientId) Retrieves the previous round’s local representation for a client.
string GetStrategyName()
void StoreRepresentation(int clientId, Vector<T> representation) Stores the current local representation for a client to be used as the negative pair in the next round.