FedProxAggregationStrategy<T>
Models & Types · AiDotNet.FederatedLearning.Aggregators
Implements the Federated Proximal (FedProx) aggregation strategy.
Inheritance: FederatedLearningComponentBase<T> → AggregationStrategyBase<Dictionary<string, T[]>, T> → ParameterDictionaryAggregationStrategyBase<T> → FedProxAggregationStrategy<T>
How It Works
FedProx is an extension of FedAvg that handles system and statistical heterogeneity in federated learning. It was proposed by Li et al. in 2020 to address challenges when clients have different computational capabilities or data distributions.
For Beginners: FedProx is like FedAvg with a “safety rope” that prevents individual clients from pulling the shared model too far in their own direction.
Key differences from FedAvg:
- Adds a proximal term to local training objective
- Prevents client models from deviating too much from global model
- Improves convergence when clients have heterogeneous data or capabilities
How FedProx works: During local training, each client minimizes: Local Loss + (μ/2) × ||w - w_global||²
where:
- Local Loss: Standard loss on client’s data
- μ (mu): Proximal term coefficient (controls constraint strength)
- w: Client’s current model weights
- w_global: Global model weights received from server
- ||w - w_global||²: Squared distance between client and global model
For example, with μ = 0.01:
- Client trains on local data
- Proximal term penalizes large deviations from global model
- If client’s data is very different, can still adapt but with limitation
- Prevents overfitting to local data distribution
When to use FedProx:
- Non-IID data (different distributions across clients)
- System heterogeneity (some clients much slower/faster)
- Want more stable convergence than FedAvg
- Stragglers problem (some clients take much longer)
Benefits:
- Better convergence on non-IID data
- More robust to stragglers
- Theoretically proven convergence guarantees
- Small computational overhead
Limitations:
- Requires tuning μ parameter
- Slightly slower local training per iteration
- May converge slower if μ is too large
Reference: Li, T., et al. (2020). “Federated Optimization in Heterogeneous Networks.” MLSys 2020.
Constructors
| Constructor | Summary |
|---|---|
FedProxAggregationStrategy(double mu) |
Initializes a new instance of the FedProxAggregationStrategy class. |
Methods
| Method | Summary |
|---|---|
Dictionary<string, T[]> Aggregate(Dictionary<int, Dictionary<string, T[]>> clientModels, Dictionary<int, double> clientWeights) |
Aggregates client models using FedProx weighted averaging. |
double GetMu() |
Gets the proximal term coefficient μ. |
string GetStrategyName() |
Gets the name of the aggregation strategy. |