ContrastivePretraining<T>
Models & Types · AiDotNet.NeuralNetworks.Tabular
Contrastive pretraining module for SAINT architecture.
For Beginners
Contrastive learning is like a “spot the difference” game:
- Take an original sample
- Create a corrupted version (swap some feature values with others)
- Train the model to tell them apart
This helps the model learn which features are important and how they relate, without needing labels. It’s especially useful when you have lots of unlabeled data.
How It Works
SAINT uses contrastive learning as a self-supervised pretraining objective. The model learns to distinguish between original samples and corrupted versions, which helps learn meaningful representations without labels.
Constructors
| Constructor | Summary |
|---|---|
ContrastivePretraining(int embeddingDim, int numFeatures, int projectionDim, double corruptionRate, double temperature) |
Initializes contrastive pretraining module. |
Properties
| Property | Summary |
|---|---|
long ParameterCount { get; } |
Gets the total parameter count. |
Methods
| Method | Summary |
|---|---|
T ComputeContrastiveLoss(Tensor<T> originalEmbeddings, Tensor<T> corruptedEmbeddings) |
Computes contrastive loss between original and corrupted embeddings. |
T ComputeDenoisingLoss(Tensor<T> reconstructed, Tensor<T> original) |
Computes denoising loss for reconstructing corrupted features. |
Tensor<T> CorruptSamples(Tensor<T> input) |
Creates corrupted samples for contrastive learning. |
Tensor<T> GetCorruptionMask(int batchSize) |
Gets the corruption mask indicating which features were corrupted. |
void ResetState() |
Resets internal state. |
void UpdateParameters(T learningRate, Tensor<T> gradWeights, Tensor<T> gradBias) |
Updates parameters. |