DataVersionControlBase<T>
Base Classes · AiDotNet.DataVersionControl
Base class for data version control implementations.
Implements: IDataVersionControl<T>
Known derived types: DataVersionControl<T>
How It Works
For Beginners: This abstract base class provides common functionality for data version control systems. It handles storage path management, hash computation for integrity verification, and data lineage tracking.
Key features:
- Path security validation
- SHA-256 hash computation for data integrity
- Dataset versioning support
- Lineage tracking for reproducibility
Methods
| Method | Summary |
|---|---|
DatasetComparison<T> CompareDatasetVersions(string datasetName, string version1Hash, string version2Hash) |
Compares two dataset versions to see what changed. |
DatasetLineage GetDatasetLineage(string datasetName, string versionHash) |
Gets the lineage information for a dataset version. |
DatasetSnapshot GetDatasetSnapshot(string snapshotName) |
Retrieves a dataset snapshot, returning only the first dataset in the snapshot. |
DatasetStatistics<T> GetDatasetStatistics(string datasetName, string versionHash) |
Gets statistics about a dataset version. |
DatasetVersion<T> GetDatasetByTag(string datasetName, string tag) |
Gets a dataset version by its tag. |
DatasetVersion<T> GetDatasetForRun(string runId) |
Gets the dataset version used by a specific training run. |
DatasetVersion<T> GetDatasetVersion(string datasetName, string versionHash) |
Retrieves a specific version of a dataset. |
DatasetVersion<T> GetLatestDatasetVersion(string datasetName) |
Gets the latest version of a dataset. |
List<DatasetVersionInfo<T>> ListDatasetVersions(string datasetName) |
Lists all versions of a dataset. |
List<string> GetRunsUsingDataset(string datasetName, string versionHash) |
Gets all training runs that used a specific dataset version. |
List<string> ListDatasets(string filter, Dictionary<string, string> tags) |
Lists all tracked datasets. |
ValueTuple<string, Dictionary<string, string>, string, DateTime> GetAllDatasetsInSnapshot(string snapshotName) |
Retrieves all datasets in a multi-dataset snapshot. |
bool VerifyDatasetIntegrity(string datasetName, string versionHash, string currentDataPath) |
Verifies that a dataset hasn’t been modified by comparing its hash. |
string ComputeDatasetHash(string dataPath) |
Computes and stores a hash of the dataset for integrity verification. |
string CreateDatasetSnapshot(string snapshotName, Dictionary<string, string> datasets, string description) |
Creates a snapshot of multiple related datasets together. |
string CreateDatasetVersion(string datasetName, string dataPath, string description, Dictionary<string, object> metadata, Dictionary<string, string> tags) |
Creates a new dataset version. |
void DeleteDatasetVersion(string datasetName, string versionHash) |
Deletes a specific dataset version. |
void LinkDatasetToRun(string datasetName, string versionHash, string runId, string modelId) |
Links a dataset version to a model training run. |
void RecordDatasetLineage(string datasetName, string versionHash, DatasetLineage lineage) |
Records metadata about how a dataset was created or transformed. |
void TagDatasetVersion(string datasetName, string versionHash, string tag) |
Tags a dataset version for easy reference. |