TabPFNBase<T>

Base Classes · AiDotNet.NeuralNetworks.Tabular

Base class for TabPFN (Prior-Fitted Networks) for tabular data.

Implements: IParameterSource<T>

Known derived types: TabPFNClassifier<T>, TabPFNRegression<T>

For Beginners

TabPFN works differently from traditional models:

  • Pre-training: Model is trained on millions of synthetic datasets
  • In-context learning: Training data becomes part of the input
  • No gradient updates: Inference only, no fine-tuning needed
  • Transformer backbone: Uses attention to learn patterns from context

The key insight is that TabPFN learns to be a “learning algorithm” itself, similar to how GPT learns to complete text.

How It Works

TabPFN is a meta-learning approach using transformers pre-trained on synthetic data. It performs in-context learning by conditioning on training examples to make predictions on test samples.

Reference: “TabPFN: A Transformer That Solves Small Tabular Classification Problems in a Second” (2022)

As in Hollmann et al. (2023), section 4, each context (training) token is the feature encoding plus the encoding of its label, each query token is the feature encoding alone, and attention is masked so every token attends only to the context tokens: the context learns from itself, and each query reads the labelled context without seeing the other queries. Without a context the blocks fall back to full self-attention over the queries.

Properties

Property Summary
int NumNumericalFeatures { get; } Gets the number of numerical features.
long ParameterCount { get; }

Methods

Method Summary
Vector<T> GetParameters() Reads every parameter in traversal order.
void ClearContext() Clears the context data.
void ResetState() Resets internal state and caches.
void SetContext(Tensor<T> features, Tensor<T> labels) Sets the context (training) data for in-context learning.
void SetParameters(Vector<T> parameters) Restores every parameter, in the order GetParameters emitted them.
void UpdateParameters(T learningRate) Updates all trainable parameters.