TabRBase<T>

Base Classes · AiDotNet.NeuralNetworks.Tabular

Base implementation of TabR, a retrieval-augmented model for tabular data.

Implements: IParameterSource<T>

Known derived types: TabRClassifier<T>, TabRRegression<T>

For Beginners

TabR is like having a photographic memory for training data.

Architecture overview:

  1. Feature Encoder: MLP that converts raw features to embeddings
  2. Retrieval Index: Stores embeddings of all training samples
  3. K-NN Search: Finds the K most similar training samples
  4. Context Encoder: Uses attention to aggregate neighbor information
  5. Prediction Head: Makes final prediction using combined information

Why this approach works:

  • Tabular data often has “local” structure (similar inputs → similar outputs)
  • Neural networks alone may struggle with rare patterns
  • Retrieval provides explicit “memory” of similar past examples
  • Attention learns which neighbors are most relevant

Think of it as:

  • Traditional k-NN: Just average neighbors (simple but limited)
  • TabR: Learn to encode, then learn to attend to neighbors (powerful)

How It Works

TabR combines neural networks with instance-based learning. It encodes features, retrieves similar training samples, and uses attention to aggregate neighbor information for making predictions.

The architecture is the reference yandex-research/tabular-dl-tabr``bin/tabr.py model. The encoder is x~ = Linear(d_in, d_main)(x) followed by EncoderNumBlocks residual blocks x~ = x~ + Dropout(dropout1)(Linear(d_block, d_main)(Dropout(dropout0)(Activation(Linear(d_main, d_block)(Norm(x~)))))), the first without its norm. The retrieval module follows Gorishniy et al. (2024), section 3: the target and every candidate are encoded, k = W_K(x~) (of Norm(x~) under mixer normalization); the m candidates with the smallest ||k - k_i||^2 are retrieved; their weights are softmax(-||k - k_i||^2), dropped by context_dropout; each neighbour contributes the value W_Y(y_i) + T(k - k_i), with T = LinearWithoutBias(Dropout(Activation(Linear(.)))) and W_Y the label embedding; and the predictor runs PredictorNumBlocks pre-normalized residual blocks over x~ + sum_i w_i V_i, then the head Activation(Norm(.)) whose final linear map a subclass adds. Candidates are re-encoded on every pass, so the encoder and W_K also learn through the retrieved neighbours. The neighbour labels enter only when the index was built with labels and IncludeNeighborTargets is on.

Properties

Property Summary
bool IsIndexBuilt { get; } Gets whether the retrieval index has been built.
int EmbeddingDimension { get; } Gets the embedding dimension.
int NumIndexedSamples { get; } Gets the number of samples in the retrieval index.
int NumNeighbors { get; } Gets the number of nearest neighbors to retrieve.
long ParameterCount { get; }

Methods

Method Summary
Matrix<int> GetRetrievedNeighborIndices() Gets the retrieved neighbor indices from the last forward pass.
Tensor<T> GetAttentionWeights() Gets the attention weights from the last forward pass.
Vector<T> GetParameters() Gets all trainable parameters as a single vector.
void BuildIndex(Tensor<T> features) Builds the retrieval index from training data.
void ResetState() Resets internal state.
void SetParameters(Vector<T> parameters) Restores every parameter, in the order GetParameters emitted them.
void UpdateParameters(T learningRate) Updates all parameters using the calculated gradients.