ZeroShardedOptimizer

Models & Types · AiDotNet.Tensors.Engines.Optimization.Optimizers

No summary documentation available yet.

Implements: IShardedOptimizer

Constructors

Constructor Summary
ZeroShardedOptimizer(OptimizerBase inner, IProcessGroup processGroup)
ZeroShardedOptimizer(OptimizerBase inner, int rank, int worldSize)

Properties

Property Summary
IReadOnlyList<ParamGroup> ParamGroups { get; }
IReadOnlyList<int> LocalParamIds { get; }
int Rank { get; }
int WorldSize { get; }

Methods

Method Summary
OptimizerStateDict LocalStateDict()
OptimizerStateDict StateDict()
ParamGroup AddParamGroup(IDictionary<string, double> overrides)
void LoadShardedStateDict(IReadOnlyList<OptimizerStateDict> perRankShards)
void LoadStateDict(OptimizerStateDict state)
void Step()
void Step(IReadOnlyDictionary<Tensor<float>, Tensor<float>> gradients)
void ZeroGrad()