SAINTNetwork<T>
Models & Types · AiDotNet.NeuralNetworks.Tabular
SAINT (Self-Attention and Intersample Attention Transformer) neural network for tabular data.
Inheritance: NeuralNetworkBase<T> → TabularNeuralNetworkBase<T> → SAINTNetwork<T>
For Beginners
SAINT is powerful because it learns two things:
Architecture:
- Column Attention: Which features are related to each other?
- Self-attention within each sample across features
- Row Attention: Which training samples are similar?
- Inter-sample attention comparing samples in the batch
- Alternating Layers: Column and row attention alternate through the network
- MLP Head: Final prediction layers
Key insight: By comparing samples within a batch, SAINT can leverage patterns that similar samples share, making it especially effective for semi-supervised learning and when samples have meaningful relationships.
Example flow: Features [batch, num_features] → Embedding [batch, num_features, embed_dim] → Column Attention (features attend to each other) → Row Attention (samples attend to each other) → Repeat for N layers → MLP → Prediction
How It Works
SAINT combines two types of attention for tabular learning:
- Self-attention over features (column attention, like FT-Transformer)
- Inter-sample attention (row attention, comparing samples within a batch)
This implementation follows the standard neural network architecture pattern with:
- Proper inheritance from NeuralNetworkBase
- Layer-based architecture using ILayer components
- Engine-based tensor operations for CPU/GPU acceleration
- Full autodiff and JIT compilation support
Reference: “SAINT: Improved Neural Networks for Tabular Data via Row Attention and Contrastive Pre-Training” (2021)
Example
var architecture = new NeuralNetworkArchitecture<float>(inputFeatures: 8, outputSize: 4);
var options = new SAINTOptions<double> { NumFeatures = 20, EmbeddingDimension = 32, NumHeads = 8, NumLayers = 6 };
var input = Tensor<float>.CreateRandom(new[] { 1, 20 });
var trainX = Tensor<float>.CreateRandom(4, 8);
var trainY = Tensor<float>.CreateRandom(4, 2);
var result = new AiModelBuilder<float, Tensor<float>, Tensor<float>>()
.ConfigureModel(new SAINTNetwork<float>(architecture))
.Build(trainX, trainY);
var output = result.Predict(input);
Constructors
| Constructor | Summary |
|---|---|
SAINTNetwork() |
Initializes a new SAINT network with the specified architecture. |
SAINTNetwork(NeuralNetworkArchitecture<T> architecture, SAINTOptions<T> options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer, ILossFunction<T> lossFunction) |
Properties
| Property | Summary |
|---|---|
SAINTOptions<T> Options { get; } |
Gets the SAINT-specific options. |
bool UseIntersampleAttention { get; } |
Gets whether inter-sample (row) attention is enabled. |
int EmbeddingDimension { get; } |
Gets the embedding dimension. |
int NumHeads { get; } |
Gets the number of attention heads. |
int NumLayers { get; } |
Gets the number of transformer layers. |
Methods
| Method | Summary |
|---|---|
Dictionary<string, T> GetFeatureImportance() |
Gets the learned feature importance from the attention layers. |
ModelMetadata<T> GetModelMetadata() |