FedBNAggregationStrategy<T>
Models & Types · AiDotNet.FederatedLearning.Aggregators
Implements the Federated Batch Normalization (FedBN) aggregation strategy.
Inheritance: FederatedLearningComponentBase<T> → AggregationStrategyBase<Dictionary<string, T[]>, T> → ParameterDictionaryAggregationStrategyBase<T> → FedBNAggregationStrategy<T>
How It Works
FedBN is a specialized aggregation strategy that handles batch normalization layers differently from other layers in neural networks. Proposed by Li et al. in 2021, it addresses the challenge of non-IID data by keeping batch normalization parameters local.
For Beginners: FedBN recognizes that some parts of a neural network should remain personalized to each client rather than being averaged globally.
The key insight:
- Batch Normalization (BN) layers learn statistics specific to each client’s data
- Averaging BN parameters across clients with different data distributions hurts performance
- Solution: Keep BN layers local, only aggregate other layers (Conv, FC, etc.)
How FedBN works:
- During aggregation, identify batch normalization layers
- Aggregate only non-BN layers using weighted averaging
- Keep each client’s BN layers unchanged (personalized)
- Send back global model with client-specific BN layers
For example, in a CNN with layers:
- Conv1 (filters) → BN1 (normalization) → ReLU → Conv2 → BN2 → FC (classification)
FedBN aggregates:
- ✓ Conv1 filters: Averaged across clients
- ✗ BN1 params: Kept local to each client
- ✓ Conv2 filters: Averaged across clients
- ✗ BN2 params: Kept local to each client
- ✓ FC weights: Averaged across clients
Why this matters:
- Different clients may have different data ranges, distributions
- Hospital A images: brightness range [0, 100]
- Hospital B images: brightness range [50, 200]
- Each needs different normalization parameters
- Shared feature extractors (Conv layers) + personalized normalization works better
When to use FedBN:
- Training deep neural networks (especially CNNs)
- Non-IID data with distribution shift
- Batch normalization or layer normalization in architecture
- Want to improve accuracy without changing training much
Benefits:
- Significantly improves accuracy on non-IID data
- Simple modification to FedAvg
- No additional communication cost
- Each client keeps personalized normalization
Limitations:
- Only helps when using batch normalization
- Doesn’t address other heterogeneity challenges
- Requires identifying BN layers in model structure
Reference: Li, X., et al. (2021). “Federated Learning on Non-IID Data Silos: An Experimental Study.” ICDE 2021.
Constructors
| Constructor | Summary |
|---|---|
FedBNAggregationStrategy(HashSet<string> batchNormLayerPatterns) |
Initializes a new instance of the FedBNAggregationStrategy class. |
Methods
| Method | Summary |
|---|---|
Dictionary<string, T[]> Aggregate(Dictionary<int, Dictionary<string, T[]>> clientModels, Dictionary<int, double> clientWeights) |
Aggregates client models while keeping batch normalization layers local. |
IReadOnlyCollection<string> GetBatchNormPatterns() |
Gets the batch normalization layer patterns used for identification. |
string GetStrategyName() |
Gets the name of the aggregation strategy. |