IMomentumEncoder<T>
Interfaces · AiDotNet.SelfSupervisedLearning
Defines the contract for momentum-updated encoders used in SSL methods.
Implements: IParameterSource<T>
Known implementations: MomentumEncoder<T>
For Beginners
A momentum encoder is a copy of the main encoder that updates more slowly using exponential moving average (EMA). This provides stable, consistent targets during training.
How It Works
How it works:
Where m is typically 0.99-0.999 (very slow updates).
Why use a momentum encoder?
Used by: MoCo, MoCo v2, MoCo v3, BYOL, DINO
Not used by: SimCLR, SimSiam (uses stop-gradient instead), Barlow Twins
Properties
| Property | Summary |
|---|---|
INeuralNetwork<T> Encoder { get; } |
Gets the underlying momentum-updated encoder network. |
double Momentum { get; } |
Gets the momentum coefficient for EMA updates. |
Methods
| Method | Summary |
|---|---|
Tensor<T> Encode(Tensor<T> input) |
Encodes input using the momentum encoder (no gradient computation). |
void CopyFromMainEncoder(INeuralNetwork<T> mainEncoder) |
Copies all parameters from the main encoder (hard copy, not EMA). |
void SetMomentum(double momentum) |
Sets the momentum coefficient. |
void UpdateFromMainEncoder(INeuralNetwork<T> mainEncoder) |
Updates the momentum encoder parameters using EMA from the main encoder. |
void UpdateFromParameters(Vector<T> mainEncoderParams) |
Updates the momentum encoder parameters using EMA from parameter vectors. |