CommunicationBackendBase<T>

Base Classes · AiDotNet.DistributedTraining

Provides base implementation for distributed communication backends.

Implements: IAsyncCommunicationBackend<T>

Known derived types: GlooCommunicationBackend<T>, InMemoryCommunicationBackend<T>, MPICommunicationBackend<T>, NCCLCommunicationBackend<T>, SharedMemoryCommunicationBackend<T>

For Beginners

This is the foundation that all communication systems build upon.

Think of this as a template that defines how any communication system should work. It handles common tasks like:

  • Keeping track of whether the system is initialized
  • Validating inputs (checking for null values, correct sizes, etc.)
  • Providing helper methods and background FIFO queuing for overlapping communication with computation

Specific communication backends (like MPI, Gloo, shared-memory, or in-memory) inherit from this and add their own implementation details. This prevents code duplication and ensures all backends work consistently.

How It Works

This abstract class implements common functionality for all communication backends, including state management, validation, and helper methods for collective operations. Derived classes implement the specific communication mechanisms (MPI, NCCL, shared-memory, in-memory, etc.).

Properties

Property Summary
bool IsInitialized { get; }
double TotalCollectiveMilliseconds { get; }
int Rank { get; }
int WorldSize { get; }
long CollectiveCount { get; }

Methods

Method Summary
IAsyncCollectiveHandle AllReduceAsync(Vector<T> data, ReductionOperation operation)
IAsyncCollectiveResultHandle<T> AllGatherAsync(Vector<T> sendData)
IAsyncCollectiveResultHandle<T> ReduceScatterAsync(Vector<T> data, ReductionOperation operation)
Vector<T> AllGather(Vector<T> sendData)
Vector<T> Broadcast(Vector<T> data, int root)
Vector<T> Receive(int sourceRank, int count, int tag)
Vector<T> ReduceScatter(Vector<T> data, ReductionOperation operation)
Vector<T> Scatter(Vector<T> sendData, int root)
void AllReduce(Vector<T> data, ReductionOperation operation)
void Barrier()
void Initialize()
void ResetDiagnostics()
void Send(Vector<T> data, int destinationRank, int tag)
void Shutdown()