MoCoV3<T>
Models & Types · AiDotNet.SelfSupervisedLearning
MoCo v3: An Empirical Study of Training Self-Supervised Vision Transformers.
Inheritance: ModelBase<T, Tensor<T>, Tensor<T>> → SelfSupervisedLearningMethodBase<T> → MoCoV3<T>
For Beginners
MoCo v3 adapts momentum contrastive learning specifically for Vision Transformers (ViT). It simplifies the framework by removing the memory queue and using in-batch negatives with a symmetric loss.
How It Works
Key changes from MoCo v1/v2:
Training stability for ViT:
Reference: Chen et al., “An Empirical Study of Training Self-Supervised Vision Transformers” (ICCV 2021)
Best for: Vision Transformers (ViT), modern architectures.
Pros: Optimized for ViT, simpler than MoCo v1/v2.
Cons: Best suited for transformer architectures.
Constructors
| Constructor | Summary |
|---|---|
MoCoV3(INeuralNetwork<T> encoder, IMomentumEncoder<T> momentumEncoder, IProjectorHead<T> projector, IProjectorHead<T> momentumProjector, IProjectorHead<T> predictor, SelfSupervisedLearningConfig<T> config) |
Initializes a new instance of the MoCoV3 class. |
Properties
| Property | Summary |
|---|---|
SelfSupervisedLearningMethodCategory Category { get; } |
|
bool RequiresMemoryBank { get; } |
|
bool UsesMomentumEncoder { get; } |
|
string Name { get; } |
Methods
| Method | Summary |
|---|---|
MoCoV3<T> Create(INeuralNetwork<T> encoder, Func<INeuralNetwork<T>, INeuralNetwork<T>> createEncoderCopy, int encoderOutputDim, int projectionDim, int hiddenDim) |
Creates a MoCo v3 instance with default configuration. |
void OnEpochStart(int epochNumber) |