DINOLoss<T>
Models & Types · AiDotNet.SelfSupervisedLearning.Losses
DINO (Self-Distillation with No Labels) Loss for self-supervised learning.
Inheritance: ContrastiveLossBase<T> → DINOLoss<T>
For Beginners
DINO loss is a cross-entropy loss between student and teacher outputs, where the teacher is an EMA of the student. It uses centering and sharpening to prevent collapse.
How It Works
Key components:
Loss formula:
Reference: Caron et al., “Emerging Properties in Self-Supervised Vision Transformers” (ICCV 2021)
Constructors
| Constructor | Summary |
|---|---|
DINOLoss(int outputDim, double studentTemperature, double teacherTemperature, double centerMomentum) |
Initializes a new instance of the DINOLoss class. |
Properties
| Property | Summary |
|---|---|
double StudentTemperature { get; } |
Gets the student temperature parameter. |
double TeacherTemperature { get; } |
Gets the teacher temperature parameter. |
Methods
| Method | Summary |
|---|---|
T ComputeLoss(Tensor<T> studentOutput, Tensor<T> teacherOutput, bool updateCenter) |
Computes the DINO loss between student and teacher outputs. |
T ComputeMultiCropLoss(IList<Tensor<T>> studentOutputs, IList<Tensor<T>> teacherOutputs) |
Computes DINO loss for multiple student crops against global teacher views. |
T[] GetCenter() |
Gets the current center values. |
Tensor<T> ComputeDifferentiableLoss(Tensor<T> view1, Tensor<T> view2, bool updateCenter) |
Differentiable DINO cross-entropy: -sum(P_teacher * log P_student). |
Tensor<T> ComputeLoss(Tensor<T> view1, Tensor<T> view2) |
|
ValueTuple<T, Tensor<T>> ComputeLossWithGradients(Tensor<T> studentOutput, Tensor<T> teacherOutput) |
Computes DINO loss with gradients for backpropagation. |
void ResetCenter() |
Resets the center to zeros. |