TapeTrainingStep<T>

Helpers & Utilities · AiDotNet.Training

Provides PyTorch-style training step using tape-based automatic differentiation, with two-level caching for parameter collection that outperforms PyTorch’s model.parameters() which rebuilds the full list on every call.

How It Works

Performance advantages over PyTorch:

Both caches are invalidated via a version counter that increments when the layer structure changes (layers added/removed).

Methods

Method Summary
IReadOnlyList<Tensor<T>> CollectParameters(IEnumerable<ILayer<T>> layers, int structureVersion) Collects all trainable parameters by recursively walking layers and their sub-layers.
ITrainableLayer<T>[] CollectTrainableLayers(IEnumerable<ILayer<T>> layers, int structureVersion) Collects all ITrainableLayer instances for ZeroGrad.
T Step(IReadOnlyList<ITrainableLayer<T>> layers, Tensor<T> input, Tensor<T> target, T learningRate, Func<Tensor<T>, Tensor<T>> forward, Func<Tensor<T>, Tensor<T>, Tensor<T>> computeLoss, Action<IReadOnlyDictionary<Tensor<T>, Tensor<T>>> onGradients) Executes a single training step using tape-based autodiff.
void InvalidateCache() Invalidates all caches.
void ZeroGradAll(IEnumerable<ILayer<T>> layers, int structureVersion) Zeros gradients for all trainable layers, including those nested in composite layers.