IDataVersionControl

Interfaces · AiDotNet.DataVersioning

Interface for data version control systems.

Implements: IDisposable

Known implementations: DataVersionControl

How It Works

For Beginners: Data version control (like DVC) helps you track changes to your datasets, ensuring you can always reproduce experiments with the exact same data.

Key concepts:

  • Dataset: A collection of data files (training data, validation data, etc.)
  • Version: A specific snapshot of a dataset at a point in time
  • Hash: A unique identifier based on file contents
  • Lineage: The history of how data was transformed

Properties

Property Summary
string StorageDirectory { get; } Gets the storage directory for data versions.

Methods

Method Summary
DataLineage GetLineage(string datasetId, string versionId) Gets the lineage (data ancestry) for a version.
DataVersion AddVersion(string datasetId, string sourcePath, string message, Dictionary<string, string> metadata) Adds a new version of a dataset.
DataVersion GetVersion(string datasetId, string versionId) Gets a specific version of a dataset.
DataVersionDiff CompareVersions(string datasetId, string versionId1, string versionId2) Compares two versions of a dataset.
List<DataVersion> ListVersions(string datasetId) Lists all versions of a dataset.
List<DatasetInfo> ListDatasets() Lists all registered datasets.
string CreateDataset(string name, string description, Dictionary<string, string> metadata) Registers a new dataset for version control.
string GetDataPath(string datasetId, string versionId) Gets the path to access a specific version’s data.
void DeleteDataset(string datasetId) Deletes a dataset and all its versions.
void DeleteVersion(string datasetId, string versionId) Deletes a specific version.
void RecordLineage(string outputDatasetId, string outputVersionId, List<ValueTuple<string, string>> inputDatasets, string transformation, Dictionary<string, object> parameters) Records data lineage (transformation history).