MPICommunicationBackend<T>
Models & Types ยท AiDotNet.DistributedTraining
MPI.NET-based communication backend for production distributed training.
Inheritance: CommunicationBackendBase<T> โ MPICommunicationBackend<T>
How It Works
Overview: MPI (Message Passing Interface) is the industry-standard communication framework for high-performance computing. MPI.NET provides .NET bindings for MPI, enabling production-grade distributed training on HPC clusters and supercomputers.
Features:
- Optimized collective operations (AllReduce, AllGather, etc.)
- Support for InfiniBand and other high-speed interconnects
- Battle-tested in HPC for decades
- Excellent performance and scalability
Use Cases:
- HPC cluster deployment
- Large-scale training (100s-1000s of nodes)
- InfiniBand or high-speed network infrastructure
- Production distributed training pipelines
Requirements:
- MPI.NET NuGet package
- MPI implementation (OpenMPI, MPICH, Intel MPI, etc.)
- MPI runtime environment
Graceful Degradation: If MPI.NET is not available, this backend falls back to single-process mode where all operations work correctly but without actual inter-process communication. A warning is logged when fallback mode is active.
Constructors
| Constructor | Summary |
|---|---|
MPICommunicationBackend(int rank, int worldSize) |
Creates a new MPI communication backend. |
Properties
| Property | Summary |
|---|---|
int Rank { get; } |
|
int WorldSize { get; } |
Methods
| Method | Summary |
|---|---|
Vector<T> AllGather(Vector<T> sendData) |
|
Vector<T> Broadcast(Vector<T> data, int root) |
|
Vector<T> Receive(int sourceRank, int count, int tag) |
|
Vector<T> ReduceScatter(Vector<T> data, ReductionOperation operation) |
|
Vector<T> Scatter(Vector<T> sendData, int root) |
|
void AllReduce(Vector<T> data, ReductionOperation operation) |
|
void Barrier() |
|
void Send(Vector<T> data, int destinationRank, int tag) |