TabTransformerBase<T>

Base Classes · AiDotNet.NeuralNetworks.Tabular

Base implementation of TabTransformer for tabular data.

Implements: IParameterSource<T>

Known derived types: TabTransformerClassifier<T>, TabTransformerRegression<T>

For Beginners

TabTransformer is like FT-Transformer but treats categories specially:

Architecture:

  1. Categorical Path: Embedding → Column Embedding → Transformer → Flatten
  2. Numerical Path: Pass through unchanged
  3. Concatenation: Combine both paths
  4. MLP Head: Final prediction layers

Key insight: Categorical features often have interactions that matter (e.g., “New York” + “Finance” vs “New York” + “Farming”). The transformer learns these relationships through self-attention.

Example flow: Categories [batch, num_cat] → Embeddings [batch, num_cat, embed_dim] → Transformer [batch, num_cat, embed_dim] → Flatten [batch, num_cat * embed_dim] ↘ Numericals [batch, num_num] → Concat [batch, num_cat * embed_dim + num_num] → MLP → Prediction

How It Works

TabTransformer applies transformer self-attention to categorical features while passing numerical features through directly. This captures complex relationships between categorical features that simple embeddings might miss.

Properties

Property Summary
int CombinedDimension { get; } Gets the combined feature dimension after concatenation.
int EmbeddingDimension { get; } Gets the embedding dimension.
int NumLayers { get; } Gets the number of transformer layers.
long ParameterCount { get; }

Methods

Method Summary
Vector<T> GetParameters()
void ResetState() Resets internal state.
void SetParameters(Vector<T> parameters)
void UpdateParameters(T learningRate) Updates all parameters using the calculated gradients.