MultiTaskEvaluationHarness<T>
Models & Types · AiDotNet.Finance.Evaluation
Multi-task evaluation harness for time series foundation models, supporting standardized evaluation across forecasting, anomaly detection, classification, imputation, and embedding tasks.
For Beginners
Foundation models like MOMENT support multiple tasks with one model. This harness lets you evaluate how well a model performs across all its supported tasks using standardized metrics for each:
- Forecasting: MSE, MAE, RMSE, MASE
- Anomaly Detection: Precision, Recall, F1-Score
- Classification: Accuracy, F1-Score
- Imputation: MSE on imputed values
- Embedding: Silhouette score for clustering quality
Constructors
| Constructor | Summary |
|---|---|
MultiTaskEvaluationHarness() |
Methods
| Method | Summary |
|---|---|
Dictionary<string, Dictionary<string, double>> RunFullEvaluation(ITimeSeriesFoundationModel<T> model, IReadOnlyList<ValueTuple<Tensor<T>, Tensor<T>>> forecastingData, ValueTuple<IReadOnlyList<Tensor<T>>, IReadOnlyList<Tensor<T>>, double?>? anomalyData, ValueTuple<IReadOnlyList<Tensor<T>>, IReadOnlyList<int>, int>? classificationData, ValueTuple<IReadOnlyList<Tensor<T>>, IReadOnlyList<Tensor<T>>>? imputationData) |
Runs a full multi-task evaluation across all supported tasks. |
Dictionary<string, double> EvaluateAnomalyDetection(ITimeSeriesFoundationModel<T> model, IReadOnlyList<Tensor<T>> testSeries, IReadOnlyList<Tensor<T>> groundTruthLabels, double? threshold) |
Evaluates a foundation model’s anomaly detection performance. |
Dictionary<string, double> EvaluateClassification(ITimeSeriesFoundationModel<T> model, IReadOnlyList<Tensor<T>> testSeries, IReadOnlyList<int> trueLabels, int numClasses) |
Evaluates a foundation model’s classification performance. |
Dictionary<string, double> EvaluateForecasting(ITimeSeriesFoundationModel<T> model, IReadOnlyList<ValueTuple<Tensor<T>, Tensor<T>>> testPairs) |
Evaluates a foundation model’s forecasting performance. |
Dictionary<string, double> EvaluateImputation(ITimeSeriesFoundationModel<T> model, IReadOnlyList<Tensor<T>> completeSeries, IReadOnlyList<Tensor<T>> masks) |
Evaluates a foundation model’s imputation performance. |