PipelineOperationType
Enums · AiDotNet.Interfaces
Types of pipeline operations.
How It Works
Traditional schedules (GPipe, 1F1B) use Forward and Backward. Zero Bubble schedules decompose Backward into BackwardInput + BackwardWeight to enable filling pipeline bubbles with weight gradient computation.
Reference: Qi et al., “Zero Bubble Pipeline Parallelism”, ICLR 2024.
https://arxiv.org/abs/2401.10241
Fields
| Field | Summary |
|---|---|
PipelineOperationType Backward |
Combined backward pass (gradient computation) through the stage’s layers. |
PipelineOperationType BackwardInput |
Backward pass computing only activation gradients (dL/dInput). |
PipelineOperationType BackwardWeight |
Backward pass computing only weight gradients (dL/dWeights). |
PipelineOperationType Forward |
Forward pass through the stage’s layers. |
int value__ |