FullyShardedDataParallel<T>
Models & Types · AiDotNet.Tensors.Engines.Distributed
No summary documentation available yet.
Constructors
| Constructor |
Summary |
FullyShardedDataParallel(IProcessGroup group, FsdpOptions options) |
|
Properties
| Property |
Summary |
FsdpOptions Options { get; } |
|
IProcessGroup Group { get; } |
|
int ParameterCount { get; } |
|
Methods
| Method |
Summary |
ShardedParameter<T> RegisterParameter(Tensor<T> fullParam) |
|
Tensor<T> GatherParameter(ShardedParameter<T> param) |
|
Tensor<T> ReduceScatterGradients(ShardedParameter<T> param, Tensor<T> fullGrad) |
|
void OptimizerStep(ShardedParameter<T> param, Tensor<T> fullParam, Tensor<T> localGradOrFullGrad, Action<Tensor<T>, Tensor<T>> step) |
|
void ReleaseParameter(ShardedParameter<T> param) |
|