FullyShardedDataParallel<T>

Models & Types · AiDotNet.Tensors.Engines.Distributed

No summary documentation available yet.

Constructors

Constructor Summary
FullyShardedDataParallel(IProcessGroup group, FsdpOptions options)

Properties

Property Summary
FsdpOptions Options { get; }
IProcessGroup Group { get; }
int ParameterCount { get; }

Methods

Method Summary
ShardedParameter<T> RegisterParameter(Tensor<T> fullParam)
Tensor<T> GatherParameter(ShardedParameter<T> param)
Tensor<T> ReduceScatterGradients(ShardedParameter<T> param, Tensor<T> fullGrad)
void OptimizerStep(ShardedParameter<T> param, Tensor<T> fullParam, Tensor<T> localGradOrFullGrad, Action<Tensor<T>, Tensor<T>> step)
void ReleaseParameter(ShardedParameter<T> param)