SelfSupervisedLearningDistributedConfig

Options & Configuration · AiDotNet.SelfSupervisedLearning

Configuration for distributed SSL training using DDP (Distributed Data Parallel).

For Beginners

This configuration enables training across multiple GPUs or machines. DDP (Distributed Data Parallel) is the industry-standard approach used by PyTorch, TensorFlow, and JAX for distributed training.

How It Works

How DDP works for SSL:

SSL-specific benefits:

Constructors

Constructor Summary
SelfSupervisedLearningDistributedConfig()

Properties

Property Summary
SelfSupervisedLearningCommunicationBackend Backend { get; set; } Gets or sets the communication backend type.
bool Enabled { get; set; } Gets or sets whether distributed training is enabled.
bool FindUnusedParameters { get; set; } Gets or sets whether to use find_unused_parameters behavior.
bool SharedMemoryQueue { get; set; } Gets or sets whether all workers share the same memory queue (for MoCo).
bool SyncBatchNorm { get; set; } Gets or sets whether to synchronize BatchNorm statistics across workers.
bool UseGradientCompression { get; set; } Gets or sets whether to use gradient compression for communication.
int GradientSyncFrequency { get; set; } Gets or sets the gradient synchronization frequency.
int Rank { get; set; } Gets or sets the rank of this worker (0-indexed).
int WorldSize { get; set; } Gets or sets the number of workers (GPUs/processes) for distributed training.

Methods

Method Summary
IDictionary<string, object> GetConfiguration() Gets the configuration as a dictionary.