TrainingMonitor<T>
Models & Types ยท AiDotNet.TrainingMonitoring
Implementation of training monitoring system for tracking model training progress.
Inheritance: TrainingMonitorBase<T> โ TrainingMonitor<T>
How It Works
For Beginners: This is a complete implementation of a training monitor that tracks all aspects of your model training in real-time.
Features include:
- Real-time metric logging (loss, accuracy, etc.)
- Resource usage tracking (CPU, memory, GPU)
- Progress estimation and ETA calculation
- Automatic issue detection (NaN values, stalled training, etc.)
- Export to JSON/CSV formats
Constructors
| Constructor | Summary |
|---|---|
TrainingMonitor() |
Initializes a new instance of the TrainingMonitor class. |
Methods
| Method | Summary |
|---|---|
Dictionary<string, T> GetCurrentMetrics(string sessionId) |
Gets the current metrics for a session. |
List<ValueTuple<int, T, DateTime>> GetMetricHistory(string sessionId, string metricName) |
Gets the history of a specific metric. |
List<string> CheckForIssues(string sessionId) |
Checks for potential training issues and returns warnings. |
ResourceUsageStats GetResourceUsage(string sessionId) |
Gets the current resource usage. |
TrainingSpeedStats GetSpeedStats(string sessionId) |
Gets statistics about training speed. |
string StartSession(string sessionName, Dictionary<string, object> metadata) |
Starts monitoring a training session. |
void CreateVisualization(string sessionId, List<string> metricNames, string outputPath) |
Creates a visualization of training metrics. |
void EndSession(string sessionId) |
Ends the current monitoring session. |
void ExportData(string sessionId, string filePath, string format) |
Exports monitoring data to a file. |
void LogMessage(string sessionId, LogLevel level, string message) |
Logs a text message or event during training. |
void LogMetric(string sessionId, string metricName, T value, int step, DateTime? timestamp) |
Records a metric value for the current training step. |
void LogMetrics(string sessionId, Dictionary<string, T> metrics, int step) |
Records multiple metrics at once. |
void LogResourceUsage(string sessionId, double cpuUsage, double memoryUsage, double? gpuUsage, double? gpuMemory) |
Records system resource usage. |
void OnEpochEnd(string sessionId, int epochNumber, Dictionary<string, T> metrics, TimeSpan duration) |
Records the end of a training epoch with summary metrics. |
void OnEpochStart(string sessionId, int epochNumber) |
Records the start of a new training epoch. |
void UpdateProgress(string sessionId, int currentStep, int totalSteps, int currentEpoch, int totalEpochs) |
Updates the training progress information. |