SAINTOptions<T>
Options & Configuration · AiDotNet.Models.Options
Configuration options for SAINT (Self-Attention and Intersample Attention Transformer).
Inheritance: ModelOptions → NeuralNetworkOptions → ModelHyperparameterOptions → FinancialNeuralNetworkOptions → RiskModelOptions<T> → SAINTOptions<T>
For Beginners
SAINT is powerful because it learns two things:
- Column attention: Which features are related to each other?
- Row attention: Which training samples are similar?
This dual attention makes SAINT especially good when similar samples share patterns that are useful for prediction.
Example:
How It Works
SAINT combines two types of attention:
- Self-attention over features (column attention, like FT-Transformer)
- Inter-sample attention (row attention, comparing samples within a batch)
Reference: “SAINT: Improved Neural Networks for Tabular Data via Row Attention and Contrastive Pre-Training” (2021)
Constructors
| Constructor | Summary |
|---|---|
SAINTOptions() |
Properties
| Property | Summary |
|---|---|
IActivationFunction<T> HiddenActivation { get; set; } |
Gets or sets the hidden layer activation function. |
IVectorActivationFunction<T> HiddenVectorActivation { get; set; } |
Gets or sets the hidden layer vector activation function (alternative to scalar activation). |
bool UseIntersampleAttention { get; set; } |
Gets or sets whether to use inter-sample (row) attention. |
bool UseLayerNorm { get; set; } |
Gets or sets whether to use layer normalization. |
bool UsePreNorm { get; set; } |
Gets or sets whether to use pre-norm (norm before attention) or post-norm. |
double DropoutRate { get; set; } |
Dropout rate. |
double EmbeddingInitScale { get; set; } |
Gets or sets the initialization scale for embeddings. |
int BatchSize { get; set; } |
Batch size (sequence length for inter-sample attention). |
int EmbeddingDimension { get; set; } |
Gets or sets the embedding dimension for features. |
int FeedForwardDimension { get; } |
Gets the feed-forward network dimension. |
int FeedForwardMultiplier { get; set; } |
Gets or sets the feed-forward dimension multiplier. |
int HiddenDimension { get; set; } |
Hidden dimension size for feed-forward networks. |
int NumHeads { get; set; } |
Number of attention heads. |
int NumLayers { get; set; } |
Number of transformer layers. |
int[] CategoricalCardinalities { get; set; } |
Gets or sets the cardinalities of categorical features. |
int[] MLPHiddenDimensions { get; set; } |
Gets or sets the hidden dimensions for the MLP head. |
Methods
| Method | Summary |
|---|---|
void Validate() |
Validates that the options are consistent (e.g., NumHeads evenly divides EmbeddingDimension). |