MoCoV3<T>

Models & Types · AiDotNet.SelfSupervisedLearning

MoCo v3: An Empirical Study of Training Self-Supervised Vision Transformers.

Inheritance: ModelBase<T, Tensor<T>, Tensor<T>> → SelfSupervisedLearningMethodBase<T> → MoCoV3<T>

For Beginners

MoCo v3 adapts momentum contrastive learning specifically for Vision Transformers (ViT). It simplifies the framework by removing the memory queue and using in-batch negatives with a symmetric loss.

How It Works

Key changes from MoCo v1/v2:

Training stability for ViT:

Reference: Chen et al., “An Empirical Study of Training Self-Supervised Vision Transformers” (ICCV 2021)

Best for: Vision Transformers (ViT), modern architectures.

Pros: Optimized for ViT, simpler than MoCo v1/v2.

Cons: Best suited for transformer architectures.

Constructors

Constructor Summary
MoCoV3(INeuralNetwork<T> encoder, IMomentumEncoder<T> momentumEncoder, IProjectorHead<T> projector, IProjectorHead<T> momentumProjector, IProjectorHead<T> predictor, SelfSupervisedLearningConfig<T> config) Initializes a new instance of the MoCoV3 class.

Properties

Property Summary
SelfSupervisedLearningMethodCategory Category { get; }
bool RequiresMemoryBank { get; }
bool UsesMomentumEncoder { get; }
string Name { get; }

Methods

Method Summary
MoCoV3<T> Create(INeuralNetwork<T> encoder, Func<INeuralNetwork<T>, INeuralNetwork<T>> createEncoderCopy, int encoderOutputDim, int projectionDim, int hiddenDim) Creates a MoCo v3 instance with default configuration.
void OnEpochStart(int epochNumber)