IDataVersionControl
Interfaces · AiDotNet.DataVersioning
Interface for data version control systems.
Implements: IDisposable
Known implementations: DataVersionControl
How It Works
For Beginners: Data version control (like DVC) helps you track changes to your datasets, ensuring you can always reproduce experiments with the exact same data.
Key concepts:
- Dataset: A collection of data files (training data, validation data, etc.)
- Version: A specific snapshot of a dataset at a point in time
- Hash: A unique identifier based on file contents
- Lineage: The history of how data was transformed
Properties
| Property | Summary |
|---|---|
string StorageDirectory { get; } |
Gets the storage directory for data versions. |
Methods
| Method | Summary |
|---|---|
DataLineage GetLineage(string datasetId, string versionId) |
Gets the lineage (data ancestry) for a version. |
DataVersion AddVersion(string datasetId, string sourcePath, string message, Dictionary<string, string> metadata) |
Adds a new version of a dataset. |
DataVersion GetVersion(string datasetId, string versionId) |
Gets a specific version of a dataset. |
DataVersionDiff CompareVersions(string datasetId, string versionId1, string versionId2) |
Compares two versions of a dataset. |
List<DataVersion> ListVersions(string datasetId) |
Lists all versions of a dataset. |
List<DatasetInfo> ListDatasets() |
Lists all registered datasets. |
string CreateDataset(string name, string description, Dictionary<string, string> metadata) |
Registers a new dataset for version control. |
string GetDataPath(string datasetId, string versionId) |
Gets the path to access a specific version’s data. |
void DeleteDataset(string datasetId) |
Deletes a dataset and all its versions. |
void DeleteVersion(string datasetId, string versionId) |
Deletes a specific version. |
void RecordLineage(string outputDatasetId, string outputVersionId, List<ValueTuple<string, string>> inputDatasets, string transformation, Dictionary<string, object> parameters) |
Records data lineage (transformation history). |