TensorTapeOps

Helpers & Utilities · AiDotNet.Helpers

Tape-tracked element-wise tensor operations that wrap engine ops which otherwise bypass the autodiff graph.

How It Works

TensorMultiplyScalar and TensorAddScalar are not recorded on the autodiff tape — using them inside a layer’s Forward path leaves the downstream trainable parameters with zero gradients under tape-based training (the LayerTestBase TapeGradient assertion specifically names this as a common cause). The wrappers here construct a constant tensor of the operand’s shape and route the operation through TensorMultiply / TensorAdd, both of which are tape-connected; the constant tensor itself is not a trainable leaf, so backward only propagates gradients through the original tensor as the math demands.

Cost: one fresh constant tensor allocation per call. For hot inner loops with fixed scalar+shape pairs, lift the tensor with CreateDefault outside the loop and use TensorMultiply / TensorAdd directly. For one-off element-wise scalar ops in Forward, these wrappers are the simplest correct replacement.

Methods

Method Summary
Tensor<T> TapeAddScalar(IEngine engine, Tensor<T> tensor, T scalar) Tape-tracked replacement for TensorAddScalar: returns tensor + scalar with the autodiff tape recording the op.
Tensor<T> TapeMultiplyScalar(IEngine engine, Tensor<T> tensor, T scalar) Tape-tracked replacement for TensorMultiplyScalar: returns tensor * scalar with the autodiff tape recording the op.