InfoNCELoss<T>

Models & Types · AiDotNet.SelfSupervisedLearning.Losses

InfoNCE (Noise Contrastive Estimation) Loss for contrastive learning.

Inheritance: ContrastiveLossBase<T> → InfoNCELoss<T>

For Beginners

InfoNCE is the loss function used in MoCo (Momentum Contrast). It’s similar to NT-Xent but designed to work efficiently with a large memory queue of negative samples.

How It Works

Key differences from NT-Xent:

Loss formula:

Constructors

Constructor Summary
InfoNCELoss(double temperature, bool normalize) Initializes a new instance of the InfoNCELoss class.

Properties

Property Summary
double Temperature { get; } Gets the temperature parameter.

Methods

Method Summary
T ComputeLoss(Tensor<T> queries, Tensor<T> positiveKeys, Tensor<T> negativeKeys) Computes the InfoNCE loss using queries, positive keys, and negative keys from memory bank.
T ComputeLossInBatch(Tensor<T> queries, Tensor<T> keys) Computes InfoNCE loss with in-batch negatives (SimCLR style but asymmetric).
Tensor<T> ComputeLoss(Tensor<T> view1, Tensor<T> view2) Differentiable in-batch InfoNCE: queries against keys, positives on the diagonal.
ValueTuple<T, Tensor<T>, Tensor<T>> ComputeLossInBatchWithGradients(Tensor<T> queries, Tensor<T> keys) Computes InfoNCE loss with in-batch negatives and gradients.
ValueTuple<T, Tensor<T>, Tensor<T>> ComputeLossWithGradients(Tensor<T> queries, Tensor<T> positiveKeys, Tensor<T> negativeKeys) Computes InfoNCE loss with gradients.
double ComputeAccuracy(Tensor<T> queries, Tensor<T> positiveKeys, Tensor<T> negativeKeys) Computes accuracy of the contrastive task (useful for monitoring).