SymmetricProjector<T>

Models & Types · AiDotNet.SelfSupervisedLearning

Symmetric Projector Head for BYOL and SimSiam-style methods.

Implements: IParameterChunkSource<T>, IParameterLayoutSource, IProjectorHead<T>

For Beginners

The symmetric projector is used in BYOL and SimSiam. It consists of a projector MLP followed by a predictor MLP. The predictor creates asymmetry between online and target branches, which is key to avoiding collapse.

How It Works

Architecture:

Key insight: The predictor is only applied to the online branch, creating asymmetry. The target branch only uses the projector.

Gradients: every operation is an engine op on tensor parameters, so a training step records the projector and predictor on the gradient tape. The earlier implementation kept its weights in plain arrays and computed with hand-written loops, so no gradient could reach it, or the encoder behind it, through the tape.

Constructors

Constructor Summary
SymmetricProjector(int inputDim, int hiddenDim, int projectionDim, int predictorHiddenDim, int? seed) Initializes a new instance of the SymmetricProjector class.

Properties

Property Summary
bool HasPredictor { get; } Gets whether this projector has a predictor head.
int InputDimension { get; }
int OutputDimension { get; }
int? HiddenDimension { get; }
long ParameterCount { get; }

Methods

Method Summary
IReadOnlyList<Tensor<T>> GetTrainableTensors()
Tensor<T> Predict(Tensor<T> projection) Applies the predictor head to a projection (for the online branch only); a projector without one returns the projection.
Tensor<T> Predict(Tensor<T> projection, int branchIndex) Applies the predictor head.
Tensor<T> Project(Tensor<T> input)
Tensor<T> ProjectAndPredict(Tensor<T> input) Projects and predicts in one call (convenience method).
Vector<T> GetParameterGradients()
Vector<T> GetParameters()
void ClearGradients()
void Reset()
void SetParameters(Vector<T> parameters)
void SetTrainingMode(bool isTraining)