IExperimentTracker

Interfaces · AiDotNet.TrainingMonitoring.ExperimentTracking

Interface for experiment tracking systems.

Implements: IDisposable

Known implementations: ExperimentTracker

How It Works

For Beginners: Experiment tracking is essential for machine learning workflows. It helps you:

  • Keep track of different experiments and their parameters
  • Compare results across runs
  • Reproduce experiments by logging all relevant information
  • Organize and search through past experiments

Think of it like MLflow or Weights & Biases - it’s a central place to log everything about your training runs.

Key concepts:

  • Experiment: A named project (e.g., “image-classification”)
  • Run: A single execution of training (e.g., “run_20241220_143052”)
  • Parameters: Configuration values (learning_rate, batch_size, etc.)
  • Metrics: Measured values (loss, accuracy, etc.)
  • Artifacts: Files produced (model weights, plots, etc.)
  • Tags: Labels for organization (dev, production, etc.)

Properties

Property Summary
string ActiveExperiment { get; } Gets the active experiment name, if any.
string ActiveRunId { get; } Gets the active run ID, if any.
string TrackingUri { get; } Gets the tracking URI.

Methods

Method Summary
ExperimentInfo CreateExperiment(string name, string description, Dictionary<string, string> tags) Creates or gets an experiment by name.
ExperimentInfo GetExperiment(string name) Gets an experiment by name.
List<ExperimentInfo> ListExperiments() Lists all experiments.
List<MetricValue> GetMetricHistory(string runId, string metricKey) Gets metric history for a run.
List<RunInfo> ListRuns(string experimentName, string filter, string orderBy, int maxResults) Lists runs in an experiment.
List<RunInfo> SearchRuns(IEnumerable<string> experimentNames, string filter, string orderBy, int maxResults) Searches for runs matching criteria.
RunComparison CompareRuns(string[] runIds) Compares multiple runs.
RunInfo GetRun(string runId) Gets a run by ID.
RunInfo StartRun(string runName, Dictionary<string, string> tags, string description) Starts a new run within the active experiment.
void DeleteExperiment(string experimentName) Deletes an experiment and all its runs.
void DeleteRun(string runId) Deletes a run.
void EndRun(RunStatus status) Ends the active run.
void LogArtifact(string localPath, string artifactPath) Logs an artifact file.
void LogArtifacts(string localDir, string artifactPath) Logs all files in a directory as artifacts.
void LogMetric(string key, double value, long? step) Logs a metric at a specific step.
void LogMetrics(Dictionary<string, double> metrics, long? step) Logs multiple metrics at a specific step.
void LogModel(string modelPath, string modelName, Dictionary<string, object> metadata) Logs a model artifact.
void LogParameter(string key, string value) Logs a parameter (configuration value).
void LogParameters(Dictionary<string, string> parameters) Logs multiple parameters.
void RestoreRun(string runId) Restores a deleted run.
void SetExperiment(string name) Sets the active experiment for subsequent runs.
void SetTag(string key, string value) Sets a tag on the active run.
void SetTags(Dictionary<string, string> tags) Sets multiple tags on the active run.