BYOL<T>

Models & Types · AiDotNet.SelfSupervisedLearning

BYOL: Bootstrap Your Own Latent - Self-supervised learning without negative samples.

Inheritance: ModelBase<T, Tensor<T>, Tensor<T>> → SelfSupervisedLearningMethodBase<T> → BYOL<T>

For Beginners

BYOL is a breakthrough method that learns representations without requiring negative samples. It uses an online network that learns to predict the output of a target network, which is updated as an exponential moving average (EMA) of the online network.

How It Works

Key innovations:

Architecture:

Why it doesn’t collapse: The combination of the predictor (asymmetry), EMA updates (target moves slowly), and batch normalization prevents trivial solutions.

Reference: Grill et al., “Bootstrap Your Own Latent - A New Approach to Self-Supervised Learning” (NeurIPS 2020)

Best for: Avoiding negative sample mining, asymmetric networks.

Pros: No negative samples needed, robust to batch size.

Cons: Requires careful design to prevent collapse.

Constructors

Constructor Summary
BYOL(INeuralNetwork<T> encoder, IMomentumEncoder<T> targetEncoder, SymmetricProjector<T> onlineProjector, SymmetricProjector<T> targetProjector, SelfSupervisedLearningConfig<T> config) Initializes a new instance of the BYOL class.

Properties

Property Summary
SelfSupervisedLearningMethodCategory Category { get; }
bool RequiresMemoryBank { get; }
bool UsesMomentumEncoder { get; }
string Name { get; }

Methods

Method Summary
BYOL<T> Create(INeuralNetwork<T> encoder, Func<INeuralNetwork<T>, INeuralNetwork<T>> createEncoderCopy, int encoderOutputDim, int projectionDim, int hiddenDim) Creates a BYOL instance with default configuration.
void OnEpochStart(int epochNumber)