DataVersionControl

Models & Types · AiDotNet.DataVersioning

DVC-equivalent data version control system for ML datasets.

Implements: IDataVersionControl

How It Works

For Beginners: This class provides Git-like version control for your datasets. It tracks changes, maintains history, and ensures reproducibility.

Key features:

  • Content-addressable storage (files identified by hash)
  • Efficient deduplication (same content stored once)
  • Full version history with diff capabilities
  • Data lineage tracking
  • Metadata and tagging support

Example usage:

Constructors

Constructor Summary
DataVersionControl(string storageDirectory) Initializes a new instance of the DataVersionControl class.

Properties

Property Summary
string StorageDirectory { get; }

Methods

Method Summary
DataLineage GetLineage(string datasetId, string versionId)
DataVersion AddVersion(string datasetId, string sourcePath, string message, Dictionary<string, string> metadata)
DataVersion GetVersion(string datasetId, string versionId)
DataVersionDiff CompareVersions(string datasetId, string versionId1, string versionId2)
List<DataVersion> ListVersions(string datasetId)
List<DatasetInfo> ListDatasets()
string CreateDataset(string name, string description, Dictionary<string, string> metadata)
string GetDataPath(string datasetId, string versionId)
void DeleteDataset(string datasetId)
void DeleteVersion(string datasetId, string versionId)
void Dispose() Disposes the data version control system.
void RecordLineage(string outputDatasetId, string outputVersionId, List<ValueTuple<string, string>> inputDatasets, string transformation, Dictionary<string, object> parameters)