DINO<T>
Models & Types · AiDotNet.SelfSupervisedLearning
DINO: Self-Distillation with No Labels - a self-supervised method for Vision Transformers.
Inheritance: ModelBase<T, Tensor<T>, Tensor<T>> → SelfSupervisedLearningMethodBase<T> → TeacherStudentSSL<T> → DINO<T>
For Beginners
DINO is a self-supervised method specifically designed for Vision Transformers (ViT). It learns by having a student network predict the output of a teacher network, where the teacher is an EMA of the student.
How It Works
Key innovations:
Architecture:
Reference: Caron et al., “Emerging Properties in Self-Supervised Vision Transformers” (ICCV 2021)
Best for: Vision Transformers, emergent attention properties.
Pros: Emergent attention maps, strong ViT performance.
Cons: Primarily designed for Vision Transformers.
Constructors
| Constructor | Summary |
|---|---|
DINO(INeuralNetwork<T> studentEncoder, IMomentumEncoder<T> teacherEncoder, IProjectorHead<T> studentProjector, IProjectorHead<T> teacherProjector, int outputDim, SelfSupervisedLearningConfig<T> config) |
Initializes a new instance of the DINO class. |
Properties
| Property | Summary |
|---|---|
SelfSupervisedLearningMethodCategory Category { get; } |
|
string Name { get; } |
Methods
| Method | Summary |
|---|---|
DINO<T> Create(INeuralNetwork<T> encoder, Func<INeuralNetwork<T>, INeuralNetwork<T>> createEncoderCopy, int encoderOutputDim, int projectionDim, int hiddenDim, int outputDim) |
Creates a DINO instance with default configuration. |