REaLTabFormerGenerator<T>

Models & Types · AiDotNet.NeuralNetworks.SyntheticData

REaLTabFormer generator using GPT-2 style autoregressive transformer for synthetic tabular data generation by treating columns as a sequence of tokens.

Inheritance: NeuralNetworkBase<T> → NeuralSyntheticTabularGeneratorBase<T> → REaLTabFormerGenerator<T>

Implements: ISyntheticTabularGenerator<T>

For Beginners

REaLTabFormer generates data like writing a sentence word by word:

If you provide custom layers in the architecture, those will be used directly for the FFN blocks. If not, the network creates the standard REaLTabFormer architecture based on the original research paper specifications.

Example usage:

How It Works

REaLTabFormer architecture:

  • Tokenization: Continuous values are binned; categoricals are integer-encoded
  • Embedding: Token embeddings + positional encodings for each column position
  • Transformer: Causal self-attention (each column can only see previous columns)
  • Output: Per-column classification heads predict the token for each column
  • Training: Cross-entropy loss on next-column prediction (teacher forcing)
  • Generation: Autoregressive sampling left-to-right with temperature

This implementation follows the standard neural network architecture pattern with:

  • Proper inheritance from NeuralNetworkBase
  • Layer-based architecture using ILayer components
  • Full autodiff and JIT compilation support

Reference: “REaLTabFormer” (Solatorio and Dupriez, 2023)

Constructors

Constructor Summary
REaLTabFormerGenerator() Initializes a new REaLTabFormer generator with the specified architecture.
REaLTabFormerGenerator(NeuralNetworkArchitecture<T> architecture, REaLTabFormerOptions<T> options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer, ILossFunction<T> lossFunction)

Properties

Property Summary
IReadOnlyList<ColumnMetadata> Columns { get; }
REaLTabFormerOptions<T> Options { get; } Gets the REaLTabFormer-specific options.
bool IsFitted { get; set; }

Methods

Method Summary
Dictionary<string, T> GetFeatureImportance()
Matrix<T> Generate(int numSamples, Vector<T> conditionColumn, Vector<T> conditionValue)
ModelMetadata<T> GetModelMetadata()
Task FitAsync(Matrix<T> data, IReadOnlyList<ColumnMetadata> columns, int? epochs, CancellationToken cancellationToken)
void Fit(Matrix<T> data, IReadOnlyList<ColumnMetadata> columns, int? epochs)