SymmetricProjector<T>
Models & Types · AiDotNet.SelfSupervisedLearning
Symmetric Projector Head for BYOL and SimSiam-style methods.
Implements: IParameterChunkSource<T>, IParameterLayoutSource, IProjectorHead<T>
For Beginners
The symmetric projector is used in BYOL and SimSiam. It consists of a projector MLP followed by a predictor MLP. The predictor creates asymmetry between online and target branches, which is key to avoiding collapse.
How It Works
Architecture:
Key insight: The predictor is only applied to the online branch, creating asymmetry. The target branch only uses the projector.
Gradients: every operation is an engine op on tensor parameters, so a training step records the projector and predictor on the gradient tape. The earlier implementation kept its weights in plain arrays and computed with hand-written loops, so no gradient could reach it, or the encoder behind it, through the tape.
Constructors
| Constructor | Summary |
|---|---|
SymmetricProjector(int inputDim, int hiddenDim, int projectionDim, int predictorHiddenDim, int? seed) |
Initializes a new instance of the SymmetricProjector class. |
Properties
| Property | Summary |
|---|---|
bool HasPredictor { get; } |
Gets whether this projector has a predictor head. |
int InputDimension { get; } |
|
int OutputDimension { get; } |
|
int? HiddenDimension { get; } |
|
long ParameterCount { get; } |
Methods
| Method | Summary |
|---|---|
IReadOnlyList<Tensor<T>> GetTrainableTensors() |
|
Tensor<T> Predict(Tensor<T> projection) |
Applies the predictor head to a projection (for the online branch only); a projector without one returns the projection. |
Tensor<T> Predict(Tensor<T> projection, int branchIndex) |
Applies the predictor head. |
Tensor<T> Project(Tensor<T> input) |
|
Tensor<T> ProjectAndPredict(Tensor<T> input) |
Projects and predicts in one call (convenience method). |
Vector<T> GetParameterGradients() |
|
Vector<T> GetParameters() |
|
void ClearGradients() |
|
void Reset() |
|
void SetParameters(Vector<T> parameters) |
|
void SetTrainingMode(bool isTraining) |