BarlowTwinsLoss<T>

Models & Types · AiDotNet.SelfSupervisedLearning.Losses

Barlow Twins Loss - redundancy reduction loss for self-supervised learning.

Inheritance: ContrastiveLossBase<T> → BarlowTwinsLoss<T>

For Beginners

Barlow Twins loss encourages the cross-correlation matrix between embeddings of two augmented views to be close to the identity matrix. This avoids both collapse and redundant features.

How It Works

Key insight: Instead of contrastive learning, Barlow Twins achieves representation quality through decorrelation - making different feature dimensions capture different information.

Loss components:

Loss formula:

where C is the cross-correlation matrix and λ is the redundancy weight.

Constructors

Constructor Summary
BarlowTwinsLoss(double lambda, bool normalize) Initializes a new instance of the BarlowTwinsLoss class.

Properties

Property Summary
double Lambda { get; } Gets the lambda (redundancy reduction weight) parameter.

Methods

Method Summary
T OffDiagonalSum(Tensor<T> matrix) Computes the off-diagonal sum of a matrix (useful for monitoring).
Tensor<T> ComputeCrossCorrelation(Tensor<T> z1, Tensor<T> z2, int batchSize) Computes the cross-correlation matrix between two sets of embeddings.
Tensor<T> ComputeLoss(Tensor<T> z1, Tensor<T> z2) Computes the Barlow Twins loss between two views.
ValueTuple<T, Tensor<T>, Tensor<T>> ComputeLossWithGradients(Tensor<T> z1, Tensor<T> z2) Computes the Barlow Twins loss with gradients for backpropagation.