BenchmarkResult<T>

Models & Types · AiDotNet.Reasoning.Benchmarks.Models

Results from evaluating a reasoning system on a benchmark.

For Beginners

This is like a report card for your reasoning system’s performance on a standardized test.

Key metrics:

  • Accuracy: Percentage of problems answered correctly (most important)
  • Total Evaluated: How many problems were tested
  • Correct Count: How many were answered correctly
  • Average Confidence: How confident the system was on average

Example:

Benchmark: GSM8K (Grade School Math)
Problems Evaluated: 100
Correct: 87
Accuracy: 87.0%
Average Confidence: 0.92
Average Time: 3.2 seconds per problem

This would indicate the system got 87 out of 100 math problems correct, with high confidence.

Constructors

Constructor Summary
BenchmarkResult()

Properties

Property Summary
Dictionary<string, T> AccuracyByCategory { get; set; } Breakdown of accuracy by category (if applicable).
Dictionary<string, object> Metrics { get; set; } Additional benchmark-specific metrics.
List<ProblemEvaluation<T>> ProblemResults { get; set; } Detailed results for each evaluated problem.
T Accuracy { get; set; } Overall accuracy (correct / total) as a value between 0.0 and 1.0.
T AverageConfidence { get; set; } Average confidence across all evaluated problems.
TimeSpan AverageTimePerProblem { get; } Average time per problem.
TimeSpan TotalDuration { get; set; } Total time spent evaluating all problems.
Vector<T> ConfidenceScores { get; set; } Confidence scores for each evaluated problem (as a Vector).
int CorrectCount { get; set; } Number of problems answered correctly.
int TotalEvaluated { get; set; } Total number of problems evaluated.
string BenchmarkName { get; set; } Name of the benchmark that was evaluated.

Methods

Method Summary
string GetSummary() Gets a summary string of the benchmark results.
string ToString() Returns a summary string.