ZeroShardedOptimizer
Models & Types · AiDotNet.Tensors.Engines.Optimization.Optimizers
No summary documentation available yet.
Implements: IShardedOptimizer
Constructors
| Constructor | Summary |
|---|---|
ZeroShardedOptimizer(OptimizerBase inner, IProcessGroup processGroup) |
|
ZeroShardedOptimizer(OptimizerBase inner, int rank, int worldSize) |
Properties
| Property | Summary |
|---|---|
IReadOnlyList<ParamGroup> ParamGroups { get; } |
|
IReadOnlyList<int> LocalParamIds { get; } |
|
int Rank { get; } |
|
int WorldSize { get; } |
Methods
| Method | Summary |
|---|---|
OptimizerStateDict LocalStateDict() |
|
OptimizerStateDict StateDict() |
|
ParamGroup AddParamGroup(IDictionary<string, double> overrides) |
|
void LoadShardedStateDict(IReadOnlyList<OptimizerStateDict> perRankShards) |
|
void LoadStateDict(OptimizerStateDict state) |
|
void Step() |
|
void Step(IReadOnlyDictionary<Tensor<float>, Tensor<float>> gradients) |
|
void ZeroGrad() |