DINOLoss<T>

Models & Types · AiDotNet.SelfSupervisedLearning.Losses

DINO (Self-Distillation with No Labels) Loss for self-supervised learning.

Inheritance: ContrastiveLossBase<T> → DINOLoss<T>

For Beginners

DINO loss is a cross-entropy loss between student and teacher outputs, where the teacher is an EMA of the student. It uses centering and sharpening to prevent collapse.

How It Works

Key components:

Loss formula:

Reference: Caron et al., “Emerging Properties in Self-Supervised Vision Transformers” (ICCV 2021)

Constructors

Constructor Summary
DINOLoss(int outputDim, double studentTemperature, double teacherTemperature, double centerMomentum) Initializes a new instance of the DINOLoss class.

Properties

Property Summary
double StudentTemperature { get; } Gets the student temperature parameter.
double TeacherTemperature { get; } Gets the teacher temperature parameter.

Methods

Method Summary
T ComputeLoss(Tensor<T> studentOutput, Tensor<T> teacherOutput, bool updateCenter) Computes the DINO loss between student and teacher outputs.
T ComputeMultiCropLoss(IList<Tensor<T>> studentOutputs, IList<Tensor<T>> teacherOutputs) Computes DINO loss for multiple student crops against global teacher views.
T[] GetCenter() Gets the current center values.
Tensor<T> ComputeDifferentiableLoss(Tensor<T> view1, Tensor<T> view2, bool updateCenter) Differentiable DINO cross-entropy: -sum(P_teacher * log P_student).
Tensor<T> ComputeLoss(Tensor<T> view1, Tensor<T> view2)
ValueTuple<T, Tensor<T>> ComputeLossWithGradients(Tensor<T> studentOutput, Tensor<T> teacherOutput) Computes DINO loss with gradients for backpropagation.
void ResetCenter() Resets the center to zeros.