ParameterAnalyzer<T>

Models & Types · AiDotNet.DistributedTraining

Analyzes model parameters and creates optimized groupings for distributed communication.

For Beginners

Think of ParameterAnalyzer as a smart packing assistant. When shipping items, you don’t want to send thousands of tiny packages - it’s inefficient! Instead, you group small items together into larger boxes.

How It Works

Similarly, when communicating parameters across GPUs:

  • Sending many small parameter arrays is slow (lots of communication overhead)
  • Grouping small parameters together reduces the number of messages
  • This analyzer figures out the best way to group parameters for efficiency

For example, instead of sending 1000 separate bias vectors (each with 1 parameter), we might group them into 10 larger chunks (each with 100 parameters).

Constructors

Constructor Summary
ParameterAnalyzer(int minimumGroupSize, int worldSize) Creates a new parameter analyzer with the specified settings.

Methods

Method Summary
Dictionary<string, double> CalculateDistributionStats(List<ParameterGroup<T>> groups) Calculates statistics about parameter distribution for a model.
List<ParameterGroup<T>> AnalyzeForDistribution(Vector<T> parameters) Analyzes parameters and creates groups optimized for even distribution across processes.
List<ParameterGroup<T>> AnalyzeModel(IFullModel<T, TInput, TOutput> model) Analyzes a model’s parameters and creates optimized groupings.
List<ParameterGroup<T>> AnalyzeParameters(Vector<T> parameters) Analyzes a parameter vector and creates optimized groupings.
bool ValidateGrouping(List<ParameterGroup<T>> groups, int totalParameterCount) Validates that parameter groups cover all parameters without gaps or overlaps.