DataVersionControlBase<T>

Base Classes · AiDotNet.DataVersionControl

Base class for data version control implementations.

Implements: IDataVersionControl<T>

Known derived types: DataVersionControl<T>

How It Works

For Beginners: This abstract base class provides common functionality for data version control systems. It handles storage path management, hash computation for integrity verification, and data lineage tracking.

Key features:

  • Path security validation
  • SHA-256 hash computation for data integrity
  • Dataset versioning support
  • Lineage tracking for reproducibility

Methods

Method Summary
DatasetComparison<T> CompareDatasetVersions(string datasetName, string version1Hash, string version2Hash) Compares two dataset versions to see what changed.
DatasetLineage GetDatasetLineage(string datasetName, string versionHash) Gets the lineage information for a dataset version.
DatasetSnapshot GetDatasetSnapshot(string snapshotName) Retrieves a dataset snapshot, returning only the first dataset in the snapshot.
DatasetStatistics<T> GetDatasetStatistics(string datasetName, string versionHash) Gets statistics about a dataset version.
DatasetVersion<T> GetDatasetByTag(string datasetName, string tag) Gets a dataset version by its tag.
DatasetVersion<T> GetDatasetForRun(string runId) Gets the dataset version used by a specific training run.
DatasetVersion<T> GetDatasetVersion(string datasetName, string versionHash) Retrieves a specific version of a dataset.
DatasetVersion<T> GetLatestDatasetVersion(string datasetName) Gets the latest version of a dataset.
List<DatasetVersionInfo<T>> ListDatasetVersions(string datasetName) Lists all versions of a dataset.
List<string> GetRunsUsingDataset(string datasetName, string versionHash) Gets all training runs that used a specific dataset version.
List<string> ListDatasets(string filter, Dictionary<string, string> tags) Lists all tracked datasets.
ValueTuple<string, Dictionary<string, string>, string, DateTime> GetAllDatasetsInSnapshot(string snapshotName) Retrieves all datasets in a multi-dataset snapshot.
bool VerifyDatasetIntegrity(string datasetName, string versionHash, string currentDataPath) Verifies that a dataset hasn’t been modified by comparing its hash.
string ComputeDatasetHash(string dataPath) Computes and stores a hash of the dataset for integrity verification.
string CreateDatasetSnapshot(string snapshotName, Dictionary<string, string> datasets, string description) Creates a snapshot of multiple related datasets together.
string CreateDatasetVersion(string datasetName, string dataPath, string description, Dictionary<string, object> metadata, Dictionary<string, string> tags) Creates a new dataset version.
void DeleteDatasetVersion(string datasetName, string versionHash) Deletes a specific dataset version.
void LinkDatasetToRun(string datasetName, string versionHash, string runId, string modelId) Links a dataset version to a model training run.
void RecordDatasetLineage(string datasetName, string versionHash, DatasetLineage lineage) Records metadata about how a dataset was created or transformed.
void TagDatasetVersion(string datasetName, string versionHash, string tag) Tags a dataset version for easy reference.