TabTransformerOptions<T>
Options & Configuration · AiDotNet.Models.Options
Configuration options for TabTransformer.
Inheritance: ModelOptions → NeuralNetworkOptions → ModelHyperparameterOptions → FinancialNeuralNetworkOptions → RiskModelOptions<T> → TabTransformerOptions<T>
For Beginners
TabTransformer treats categorical features specially:
Key ideas:
- Categorical Embeddings: Each category value gets a learned vector
- Transformer for Categories: Self-attention captures relationships between categories
- Numerical Features Unchanged: Numbers pass through directly
- Concatenation: Transformed categories + numbers → MLP → prediction
Why this works:
- Categorical features often have complex interactions (e.g., city + job type)
- Transformer attention can learn these interactions automatically
- Numerical features don’t need the same treatment
Example:
How It Works
TabTransformer applies transformer self-attention only to categorical features, while keeping numerical features in their original form. The transformed categorical embeddings are concatenated with numerical features for prediction.
Reference: “TabTransformer: Tabular Data Modeling Using Contextual Embeddings” (2020)
Constructors
| Constructor | Summary |
|---|---|
TabTransformerOptions() |
Initializes a new instance with default values. |
TabTransformerOptions(TabTransformerOptions<T> other) |
Initializes a new instance by copying every property from another instance. |
Properties
| Property | Summary |
|---|---|
bool UseColumnEmbedding { get; set; } |
Gets or sets whether to use column embedding (add learnable column-specific vectors). |
bool UseLayerNorm { get; set; } |
Gets or sets whether to use layer normalization. |
double DropoutRate { get; set; } |
Dropout rate. |
double EmbeddingInitScale { get; set; } |
Gets or sets the initialization scale for embeddings. |
double LearningRate { get; set; } |
Gets or sets the AdamW learning rate used when no optimizer is supplied. |
double WeightDecay { get; set; } |
Gets or sets the decoupled AdamW weight-decay coefficient used when no optimizer is supplied. |
int EmbeddingDimension { get; set; } |
Gets or sets the embedding dimension for categorical features. |
int FeedForwardDimension { get; } |
Gets the feed-forward network dimension. |
int FeedForwardMultiplier { get; set; } |
Gets or sets the feed-forward dimension multiplier. |
int HiddenDimension { get; set; } |
Hidden dimension size for transformer. |
int NumCategoricalFeatures { get; set; } |
Gets or sets the number of categorical features. |
int NumHeads { get; set; } |
Number of attention heads in each transformer block. |
int NumLayers { get; set; } |
Number of transformer layers. |
int[] CategoricalCardinalities { get; set; } |
Number of distinct values of each categorical column, in column order; null when there are no categorical columns. |
int[] MLPHiddenDimensions { get; set; } |
Gets or sets the hidden dimensions for the MLP head. |
Methods
| Method | Summary |
|---|---|
void Validate() |
Checks the relationships between these options, which individual setters cannot. |