IAsyncCollectiveHandle
Interfaces · AiDotNet.DistributedTraining
Represents an in-flight asynchronous collective communication operation.
Implements: IDisposable
Known implementations: IAsyncCollectiveResultHandle<T>
For Beginners
When training across multiple GPUs, waiting for gradients to finish copying
across devices before computing the remaining layers wastes time. An IAsyncCollectiveHandle
lets a collective operation (such as AllReduce on a finished gradient bucket) run in the background
while backward propagation continues on earlier layers, and then lets the trainer wait for completion right
before gradient unscaling and global-norm clipping.
Properties
| Property | Summary |
|---|---|
TimeSpan Elapsed { get; } |
Gets the wall-clock duration spent executing the collective operation on the transport. |
bool IsCompleted { get; } |
Gets whether the collective operation has completed. |
Methods
| Method | Summary |
|---|---|
Task WaitAsync(CancellationToken cancellationToken) |
Asynchronously waits until the collective operation completes, rethrowing any transport error. |
void Wait() |
Blocks until the collective operation completes, rethrowing any transport error. |