MPICommunicationBackend<T>

Models & Types ยท AiDotNet.DistributedTraining

MPI.NET-based communication backend for production distributed training.

Inheritance: CommunicationBackendBase<T> โ†’ MPICommunicationBackend<T>

How It Works

Overview: MPI (Message Passing Interface) is the industry-standard communication framework for high-performance computing. MPI.NET provides .NET bindings for MPI, enabling production-grade distributed training on HPC clusters and supercomputers.

Features:

  • Optimized collective operations (AllReduce, AllGather, etc.)
  • Support for InfiniBand and other high-speed interconnects
  • Battle-tested in HPC for decades
  • Excellent performance and scalability

Use Cases:

  • HPC cluster deployment
  • Large-scale training (100s-1000s of nodes)
  • InfiniBand or high-speed network infrastructure
  • Production distributed training pipelines

Requirements:

  • MPI.NET NuGet package
  • MPI implementation (OpenMPI, MPICH, Intel MPI, etc.)
  • MPI runtime environment

Graceful Degradation: If MPI.NET is not available, this backend falls back to single-process mode where all operations work correctly but without actual inter-process communication. A warning is logged when fallback mode is active.

Constructors

Constructor Summary
MPICommunicationBackend(int rank, int worldSize) Creates a new MPI communication backend.

Properties

Property Summary
int Rank { get; }
int WorldSize { get; }

Methods

Method Summary
Vector<T> AllGather(Vector<T> sendData)
Vector<T> Broadcast(Vector<T> data, int root)
Vector<T> Receive(int sourceRank, int count, int tag)
Vector<T> ReduceScatter(Vector<T> data, ReductionOperation operation)
Vector<T> Scatter(Vector<T> sendData, int root)
void AllReduce(Vector<T> data, ReductionOperation operation)
void Barrier()
void Send(Vector<T> data, int destinationRank, int tag)