ParameterAnalyzer<T>
Models & Types · AiDotNet.DistributedTraining
Analyzes model parameters and creates optimized groupings for distributed communication.
For Beginners
Think of ParameterAnalyzer as a smart packing assistant. When shipping items, you don’t want to send thousands of tiny packages - it’s inefficient! Instead, you group small items together into larger boxes.
How It Works
Similarly, when communicating parameters across GPUs:
- Sending many small parameter arrays is slow (lots of communication overhead)
- Grouping small parameters together reduces the number of messages
- This analyzer figures out the best way to group parameters for efficiency
For example, instead of sending 1000 separate bias vectors (each with 1 parameter), we might group them into 10 larger chunks (each with 100 parameters).
Constructors
| Constructor | Summary |
|---|---|
ParameterAnalyzer(int minimumGroupSize, int worldSize) |
Creates a new parameter analyzer with the specified settings. |
Methods
| Method | Summary |
|---|---|
Dictionary<string, double> CalculateDistributionStats(List<ParameterGroup<T>> groups) |
Calculates statistics about parameter distribution for a model. |
List<ParameterGroup<T>> AnalyzeForDistribution(Vector<T> parameters) |
Analyzes parameters and creates groups optimized for even distribution across processes. |
List<ParameterGroup<T>> AnalyzeModel(IFullModel<T, TInput, TOutput> model) |
Analyzes a model’s parameters and creates optimized groupings. |
List<ParameterGroup<T>> AnalyzeParameters(Vector<T> parameters) |
Analyzes a parameter vector and creates optimized groupings. |
bool ValidateGrouping(List<ParameterGroup<T>> groups, int totalParameterCount) |
Validates that parameter groups cover all parameters without gaps or overlaps. |