ContrastivePretraining<T>

Models & Types · AiDotNet.NeuralNetworks.Tabular

Contrastive pretraining module for SAINT architecture.

For Beginners

Contrastive learning is like a “spot the difference” game:

  1. Take an original sample
  2. Create a corrupted version (swap some feature values with others)
  3. Train the model to tell them apart

This helps the model learn which features are important and how they relate, without needing labels. It’s especially useful when you have lots of unlabeled data.

How It Works

SAINT uses contrastive learning as a self-supervised pretraining objective. The model learns to distinguish between original samples and corrupted versions, which helps learn meaningful representations without labels.

Constructors

Constructor Summary
ContrastivePretraining(int embeddingDim, int numFeatures, int projectionDim, double corruptionRate, double temperature) Initializes contrastive pretraining module.

Properties

Property Summary
long ParameterCount { get; } Gets the total parameter count.

Methods

Method Summary
T ComputeContrastiveLoss(Tensor<T> originalEmbeddings, Tensor<T> corruptedEmbeddings) Computes contrastive loss between original and corrupted embeddings.
T ComputeDenoisingLoss(Tensor<T> reconstructed, Tensor<T> original) Computes denoising loss for reconstructing corrupted features.
Tensor<T> CorruptSamples(Tensor<T> input) Creates corrupted samples for contrastive learning.
Tensor<T> GetCorruptionMask(int batchSize) Gets the corruption mask indicating which features were corrupted.
void ResetState() Resets internal state.
void UpdateParameters(T learningRate, Tensor<T> gradWeights, Tensor<T> gradBias) Updates parameters.