MomentumEncoder<T>

Models & Types · AiDotNet.SelfSupervisedLearning

Momentum-updated encoder for self-supervised learning methods.

Implements: IMomentumEncoder<T>

For Beginners

A momentum encoder is a copy of the main encoder that updates more slowly using exponential moving average (EMA). This provides stable, consistent targets during self-supervised training.

How It Works

Update formula:

Where m is momentum (typically 0.99-0.9999).

Why slow updates?

Example usage:

Constructors

Constructor Summary
MomentumEncoder(INeuralNetwork<T> encoder, double momentum) Initializes a new instance of the MomentumEncoder class.

Properties

Property Summary
INeuralNetwork<T> Encoder { get; }
double Momentum { get; }
long ParameterCount { get; }

Methods

Method Summary
MomentumEncoder<T> Create(TEncoder mainEncoder, double momentum, Func<TEncoder, TEncoder> cloneFunc) Creates a momentum encoder from a main encoder by cloning.
Tensor<T> Encode(Tensor<T> input)
Vector<T> GetParameters()
double ScheduleMomentum(double baseMomentum, double finalMomentum, int currentEpoch, int totalEpochs) Computes the scheduled momentum value based on training progress.
void CopyFromMainEncoder(INeuralNetwork<T> mainEncoder)
void SetMomentum(double momentum)
void SetParameters(Vector<T> parameters)
void UpdateFromMainEncoder(INeuralNetwork<T> mainEncoder)
void UpdateFromParameters(Vector<T> mainEncoderParams)