TabTransformerBase<T>
Base Classes · AiDotNet.NeuralNetworks.Tabular
Base implementation of TabTransformer for tabular data.
Implements: IParameterSource<T>
Known derived types: TabTransformerClassifier<T>, TabTransformerRegression<T>
For Beginners
TabTransformer is like FT-Transformer but treats categories specially:
Architecture:
- Categorical Path: Embedding → Column Embedding → Transformer → Flatten
- Numerical Path: Pass through unchanged
- Concatenation: Combine both paths
- MLP Head: Final prediction layers
Key insight: Categorical features often have interactions that matter (e.g., “New York” + “Finance” vs “New York” + “Farming”). The transformer learns these relationships through self-attention.
Example flow: Categories [batch, num_cat] → Embeddings [batch, num_cat, embed_dim] → Transformer [batch, num_cat, embed_dim] → Flatten [batch, num_cat * embed_dim] ↘ Numericals [batch, num_num] → Concat [batch, num_cat * embed_dim + num_num] → MLP → Prediction
How It Works
TabTransformer applies transformer self-attention to categorical features while passing numerical features through directly. This captures complex relationships between categorical features that simple embeddings might miss.
Properties
| Property | Summary |
|---|---|
int CombinedDimension { get; } |
Gets the combined feature dimension after concatenation. |
int EmbeddingDimension { get; } |
Gets the embedding dimension. |
int NumLayers { get; } |
Gets the number of transformer layers. |
long ParameterCount { get; } |
Methods
| Method | Summary |
|---|---|
Vector<T> GetParameters() |
|
void ResetState() |
Resets internal state. |
void SetParameters(Vector<T> parameters) |
|
void UpdateParameters(T learningRate) |
Updates all parameters using the calculated gradients. |