TabTransformerOptions<T>

Options & Configuration · AiDotNet.Models.Options

Configuration options for TabTransformer.

Inheritance: ModelOptions → NeuralNetworkOptions → ModelHyperparameterOptions → FinancialNeuralNetworkOptions → RiskModelOptions<T> → TabTransformerOptions<T>

For Beginners

TabTransformer treats categorical features specially:

Key ideas:

  1. Categorical Embeddings: Each category value gets a learned vector
  2. Transformer for Categories: Self-attention captures relationships between categories
  3. Numerical Features Unchanged: Numbers pass through directly
  4. Concatenation: Transformed categories + numbers → MLP → prediction

Why this works:

  • Categorical features often have complex interactions (e.g., city + job type)
  • Transformer attention can learn these interactions automatically
  • Numerical features don’t need the same treatment

Example:

How It Works

TabTransformer applies transformer self-attention only to categorical features, while keeping numerical features in their original form. The transformed categorical embeddings are concatenated with numerical features for prediction.

Reference: “TabTransformer: Tabular Data Modeling Using Contextual Embeddings” (2020)

Constructors

Constructor Summary
TabTransformerOptions() Initializes a new instance with default values.
TabTransformerOptions(TabTransformerOptions<T> other) Initializes a new instance by copying every property from another instance.

Properties

Property Summary
bool UseColumnEmbedding { get; set; } Gets or sets whether to use column embedding (add learnable column-specific vectors).
bool UseLayerNorm { get; set; } Gets or sets whether to use layer normalization.
double DropoutRate { get; set; } Dropout rate.
double EmbeddingInitScale { get; set; } Gets or sets the initialization scale for embeddings.
double LearningRate { get; set; } Gets or sets the AdamW learning rate used when no optimizer is supplied.
double WeightDecay { get; set; } Gets or sets the decoupled AdamW weight-decay coefficient used when no optimizer is supplied.
int EmbeddingDimension { get; set; } Gets or sets the embedding dimension for categorical features.
int FeedForwardDimension { get; } Gets the feed-forward network dimension.
int FeedForwardMultiplier { get; set; } Gets or sets the feed-forward dimension multiplier.
int HiddenDimension { get; set; } Hidden dimension size for transformer.
int NumCategoricalFeatures { get; set; } Gets or sets the number of categorical features.
int NumHeads { get; set; } Number of attention heads in each transformer block.
int NumLayers { get; set; } Number of transformer layers.
int[] CategoricalCardinalities { get; set; } Number of distinct values of each categorical column, in column order; null when there are no categorical columns.
int[] MLPHiddenDimensions { get; set; } Gets or sets the hidden dimensions for the MLP head.

Methods

Method Summary
void Validate() Checks the relationships between these options, which individual setters cannot.