TapeTrainingStep<T>
Helpers & Utilities · AiDotNet.Training
Provides PyTorch-style training step using tape-based automatic differentiation,
with two-level caching for parameter collection that outperforms PyTorch’s
model.parameters() which rebuilds the full list on every call.
How It Works
Performance advantages over PyTorch:
Both caches are invalidated via a version counter that increments when the layer structure changes (layers added/removed).
Methods
| Method | Summary |
|---|---|
IReadOnlyList<Tensor<T>> CollectParameters(IEnumerable<ILayer<T>> layers, int structureVersion) |
Collects all trainable parameters by recursively walking layers and their sub-layers. |
ITrainableLayer<T>[] CollectTrainableLayers(IEnumerable<ILayer<T>> layers, int structureVersion) |
Collects all ITrainableLayer instances for ZeroGrad. |
T Step(IReadOnlyList<ITrainableLayer<T>> layers, Tensor<T> input, Tensor<T> target, T learningRate, Func<Tensor<T>, Tensor<T>> forward, Func<Tensor<T>, Tensor<T>, Tensor<T>> computeLoss, Action<IReadOnlyDictionary<Tensor<T>, Tensor<T>>> onGradients) |
Executes a single training step using tape-based autodiff. |
void InvalidateCache() |
Invalidates all caches. |
void ZeroGradAll(IEnumerable<ILayer<T>> layers, int structureVersion) |
Zeros gradients for all trainable layers, including those nested in composite layers. |