StopGradient<T>

Helpers & Utilities · AiDotNet.SelfSupervisedLearning

Provides stop-gradient operations for self-supervised learning.

For Beginners

Stop-gradient (also called “detach” in PyTorch) prevents gradients from flowing through a tensor during backpropagation. This is crucial for several SSL methods:

How It Works

Why stop-gradient?

Without stop-gradient, the model could “cheat” by making both branches output constants, resulting in representation collapse. Stop-gradient forces asymmetry that prevents this.

Example usage:

Methods

Method Summary
T SymmetricLoss(Tensor<T> prediction1, Tensor<T> target1, Tensor<T> prediction2, Tensor<T> target2, Func<Tensor<T>, Tensor<T>, T> lossFunction) Computes the symmetric loss with stop-gradient for SimSiam-style training.
Tensor<T> Detach(Tensor<T> tensor) Applies stop-gradient to a vector.
Tensor<T> ZeroGrad(Tensor<T> tensor) Creates a zero-gradient version of a tensor for the backward pass.
Tensor<T>[] DetachBatch(Tensor<T>[] tensors) Detaches a batch of tensors from the computation graph.
Vector<T> Detach(Vector<T> vector) Applies stop-gradient to a vector.