StopGradient<T>
Helpers & Utilities · AiDotNet.SelfSupervisedLearning
Provides stop-gradient operations for self-supervised learning.
For Beginners
Stop-gradient (also called “detach” in PyTorch) prevents gradients from flowing through a tensor during backpropagation. This is crucial for several SSL methods:
How It Works
Why stop-gradient?
Without stop-gradient, the model could “cheat” by making both branches output constants, resulting in representation collapse. Stop-gradient forces asymmetry that prevents this.
Example usage:
Methods
| Method | Summary |
|---|---|
T SymmetricLoss(Tensor<T> prediction1, Tensor<T> target1, Tensor<T> prediction2, Tensor<T> target2, Func<Tensor<T>, Tensor<T>, T> lossFunction) |
Computes the symmetric loss with stop-gradient for SimSiam-style training. |
Tensor<T> Detach(Tensor<T> tensor) |
Applies stop-gradient to a vector. |
Tensor<T> ZeroGrad(Tensor<T> tensor) |
Creates a zero-gradient version of a tensor for the backward pass. |
Tensor<T>[] DetachBatch(Tensor<T>[] tensors) |
Detaches a batch of tensors from the computation graph. |
Vector<T> Detach(Vector<T> vector) |
Applies stop-gradient to a vector. |