BYOLLoss<T>

Models & Types · AiDotNet.SelfSupervisedLearning.Losses

BYOL (Bootstrap Your Own Latent) Loss - a simple cosine similarity loss.

Inheritance: ContrastiveLossBase<T> → BYOLLoss<T>

For Beginners

BYOL uses a simple Mean Squared Error (MSE) loss between normalized predictions and normalized targets. Unlike contrastive methods, it doesn’t require negative samples.

How It Works

Key insight: BYOL avoids collapse through asymmetric architecture (predictor on one branch) and momentum updates, not through negative samples.

Loss formula:

where p is prediction, z’ is target projection, and sg() means stop-gradient.

Constructors

Constructor Summary
BYOLLoss(bool normalize, bool symmetric) Initializes a new instance of the BYOLLoss class.

Methods

Method Summary
T ComputeMSELoss(Tensor<T> prediction, Tensor<T> target) Computes the mean squared error between normalized embeddings.
Tensor<T> ComputeLoss(Tensor<T> onlinePrediction, Tensor<T> targetProjection) Computes the BYOL loss between online predictions and target projections.
Tensor<T> ComputeSymmetricLoss(Tensor<T> pred1, Tensor<T> proj2, Tensor<T> pred2, Tensor<T> proj1) Computes symmetric BYOL loss for both views.
ValueTuple<T, Tensor<T>> ComputeLossWithGradients(Tensor<T> onlinePrediction, Tensor<T> targetProjection) Computes BYOL loss with gradients for backpropagation.