SelfSupervisedLearningDistributedConfig
Options & Configuration · AiDotNet.SelfSupervisedLearning
Configuration for distributed SSL training using DDP (Distributed Data Parallel).
For Beginners
This configuration enables training across multiple GPUs or machines. DDP (Distributed Data Parallel) is the industry-standard approach used by PyTorch, TensorFlow, and JAX for distributed training.
How It Works
How DDP works for SSL:
SSL-specific benefits:
Constructors
| Constructor | Summary |
|---|---|
SelfSupervisedLearningDistributedConfig() |
Properties
| Property | Summary |
|---|---|
SelfSupervisedLearningCommunicationBackend Backend { get; set; } |
Gets or sets the communication backend type. |
bool Enabled { get; set; } |
Gets or sets whether distributed training is enabled. |
bool FindUnusedParameters { get; set; } |
Gets or sets whether to use find_unused_parameters behavior. |
bool SharedMemoryQueue { get; set; } |
Gets or sets whether all workers share the same memory queue (for MoCo). |
bool SyncBatchNorm { get; set; } |
Gets or sets whether to synchronize BatchNorm statistics across workers. |
bool UseGradientCompression { get; set; } |
Gets or sets whether to use gradient compression for communication. |
int GradientSyncFrequency { get; set; } |
Gets or sets the gradient synchronization frequency. |
int Rank { get; set; } |
Gets or sets the rank of this worker (0-indexed). |
int WorldSize { get; set; } |
Gets or sets the number of workers (GPUs/processes) for distributed training. |
Methods
| Method | Summary |
|---|---|
IDictionary<string, object> GetConfiguration() |
Gets the configuration as a dictionary. |