TabularDataTransformer<T>
Models & Types · AiDotNet.NeuralNetworks.SyntheticData
Transforms tabular data using Variational Gaussian Mixture (VGM) mode-specific normalization for continuous columns and one-hot encoding for categorical columns. Used by CTGAN and TVAE.
Implements: IModelSerializer
For Beginners
Real-world data often has columns with very different distributions. A “price” column might have values clustered around $10 and $100 (two modes), while an “age” column might be normally distributed. Simple min-max or z-score normalization doesn’t handle multi-modal (multiple-peak) distributions well.
VGM normalization solves this by:
- Fitting a Gaussian mixture (like fitting multiple bell curves) to each continuous column
- For each value, finding which bell curve (mode) it belongs to
- Normalizing relative to that mode (so both “cheap” and “expensive” items get reasonable values)
- Adding a one-hot indicator showing which mode was chosen
This helps the generator learn multi-modal distributions much more effectively.
How It Works
The transformation follows the CTGAN paper (Xu et al., NeurIPS 2019):
- Continuous columns: A Gaussian mixture model (GMM) is fitted per column.
Each value is then represented as (normalized_value, one-hot_mode_indicator), where the normalized value is relative to the selected mode’s mean and std.
- Categorical columns: One-hot encoded into binary indicator vectors.
- Inverse transform: Reconstructs original-scale values from the transformed representation.
Constructors
| Constructor | Summary |
|---|---|
TabularDataTransformer(int vgmModes, Random random) |
Initializes a new TabularDataTransformer. |
Properties
| Property | Summary |
|---|---|
IReadOnlyList<ColumnMetadata> Columns { get; } |
Gets the column metadata this transformer was fitted on. |
bool IsFitted { get; set; } |
Gets whether this transformer has been fitted. |
int TransformedWidth { get; } |
Gets the width of the transformed data representation. |
Methods
| Method | Summary |
|---|---|
ColumnTransformInfo GetTransformInfo(int colIndex) |
Gets the transform info for a specific original column index. |
Matrix<T> InverseTransform(Matrix<T> transformed) |
Inverse-transforms generated data back to the original column space. |
Matrix<T> Transform(Matrix<T> data) |
Transforms the raw data matrix into the VGM-normalized + one-hot representation. |
byte[] Serialize() |
|
void Deserialize(BinaryReader reader) |
|
void Deserialize(byte[] data) |
|
void Fit(Matrix<T> data, IReadOnlyList<ColumnMetadata> columns) |
Fits the transformer to the data by learning VGM parameters for continuous columns and category mappings for categorical columns. |
void LoadModel(string filePath) |
|
void SaveModel(string filePath) |
|
void Serialize(BinaryWriter writer) |
Serializes the fitted transformer state (column metadata, per-column transform layout, and the fitted VGM / categorical parameters) so a saved or cloned generator can inverse-transform generated samples back to the original column space wit… |