IMomentumEncoder<T>

Interfaces · AiDotNet.SelfSupervisedLearning

Defines the contract for momentum-updated encoders used in SSL methods.

Implements: IParameterSource<T>

Known implementations: MomentumEncoder<T>

For Beginners

A momentum encoder is a copy of the main encoder that updates more slowly using exponential moving average (EMA). This provides stable, consistent targets during training.

How It Works

How it works:

Where m is typically 0.99-0.999 (very slow updates).

Why use a momentum encoder?

Used by: MoCo, MoCo v2, MoCo v3, BYOL, DINO

Not used by: SimCLR, SimSiam (uses stop-gradient instead), Barlow Twins

Properties

Property Summary
INeuralNetwork<T> Encoder { get; } Gets the underlying momentum-updated encoder network.
double Momentum { get; } Gets the momentum coefficient for EMA updates.

Methods

Method Summary
Tensor<T> Encode(Tensor<T> input) Encodes input using the momentum encoder (no gradient computation).
void CopyFromMainEncoder(INeuralNetwork<T> mainEncoder) Copies all parameters from the main encoder (hard copy, not EMA).
void SetMomentum(double momentum) Sets the momentum coefficient.
void UpdateFromMainEncoder(INeuralNetwork<T> mainEncoder) Updates the momentum encoder parameters using EMA from the main encoder.
void UpdateFromParameters(Vector<T> mainEncoderParams) Updates the momentum encoder parameters using EMA from parameter vectors.