ZeRO3Optimizer<T, TInput, TOutput>
Models & Types · AiDotNet.DistributedTraining
Implements ZeRO Stage 3 optimizer - full sharding equivalent to FSDP.
Inheritance: ShardedOptimizerBase<T, TInput, TOutput> → FSDPOptimizer<T, TInput, TOutput> → ZeRO3Optimizer<T, TInput, TOutput>
For Beginners
ZeRO-3 and FSDP optimizers are the same thing. Use whichever name you prefer. Everything is sharded for maximum memory efficiency.
How It Works
Strategy Overview: ZeRO-3 is equivalent to FSDP optimizer - full sharding of parameters, gradients, and optimizer states. This class is an alias to FSDPOptimizer for ZeRO terminology consistency.
Constructors
| Constructor | Summary |
|---|---|
ZeRO3Optimizer(IOptimizer<T, TInput, TOutput> wrappedOptimizer, IShardingConfiguration<T> config) |