AutoFsdp<T>

Models & Types · AiDotNet.Tensors.Engines.Distributed

No summary documentation available yet.

Constructors

Constructor Summary
AutoFsdp(FullyShardedDataParallel<T> fsdp, GradientTape<T> tape)

Properties

Property Summary
FullyShardedDataParallel<T> Fsdp { get; }
GradientTape<T> Tape { get; }

Methods

Method Summary
ShardedParameter<T> Shard(Tensor<T> fullParam)
Tensor<T> LocalGradient(ShardedParameter<T> sharded)
bool HasLocalGradient(ShardedParameter<T> sharded)
void ProcessGradients(IReadOnlyDictionary<Tensor<T>, Tensor<T>> grads)
void ZeroLocalGradients()