KoLeoRegularizer<T>
Models & Types · AiDotNet.ComputerVision.Segmentation.PointCloud
The Kozachenko-Leonenko entropy regulariser of DINOv2 (Sablayrolles et al. 2019):
-mean(log(||x_i - x_nn(i)|| + eps)) over L2-normalised embeddings, pushing each embedding
away from its nearest neighbour.
Implements: ISonataFeatureRegularizer<T>
How It Works
The official Sonata v1m1 configuration does not use it, so KoLeoWeight
defaults to zero; it is provided for the DINOv2-style variant.
DINOv2 picks the neighbours from the full [points, points] cosine matrix, which for a 65,536-point
view is 4.3 billion entries. Here the matrix is scanned in BlockSize-square tiles that
keep each query’s best score and global index (the first index on ties, as argmax), so memory
is bounded by one tile while the selected neighbours are the same. The selection carries no
gradient; the distance to the selected neighbour does.
Constructors
| Constructor | Summary |
|---|---|
KoLeoRegularizer(int blockSize) |
Creates the regulariser. |
Properties
| Property | Summary |
|---|---|
int BlockSize { get; } |
Rows and columns of each tile of the cosine matrix scanned for nearest neighbours. |
Methods
| Method | Summary |
|---|---|
Tensor<T> Compute(Tensor<T> embeddings) |