DistributedExtensions

Helpers & Utilities · AiDotNet.DistributedTraining

Provides extension methods for easily enabling distributed training on models and optimizers.

For Beginners

Extension methods allow you to add new functionality to existing classes without modifying them. These extensions add a simple .AsDistributed() method to models and optimizers, making it incredibly easy to enable distributed training with just one line of code!

How It Works

Example:

Methods

Method Summary
IShardedModel<T, TInput, TOutput> AsDistributed(IFullModel<T, TInput, TOutput> model, ICommunicationBackend<T> communicationBackend) Wraps an optimizer with distributed training capabilities using a custom configuration.
IShardedModel<T, TInput, TOutput> AsDistributed(IFullModel<T, TInput, TOutput> model, IShardingConfiguration<T> configuration) Wraps an optimizer with distributed training capabilities using a custom configuration.
IShardedModel<T, TInput, TOutput> AsDistributedForHighBandwidth(IFullModel<T, TInput, TOutput> model, ICommunicationBackend<T> communicationBackend) Creates a distributed model with optimized settings for high-bandwidth networks.
IShardedModel<T, TInput, TOutput> AsDistributedForLowBandwidth(IFullModel<T, TInput, TOutput> model, ICommunicationBackend<T> communicationBackend) Creates a distributed model with optimized settings for low-bandwidth networks.
IShardedOptimizer<T, TInput, TOutput> AsDistributed(IOptimizer<T, TInput, TOutput> optimizer, ICommunicationBackend<T> communicationBackend) Wraps an optimizer with distributed training capabilities using a custom configuration.
IShardedOptimizer<T, TInput, TOutput> AsDistributed(IOptimizer<T, TInput, TOutput> optimizer, IShardingConfiguration<T> configuration) Wraps an optimizer with distributed training capabilities using a custom configuration.