BenchmarkProblem
Models & Types · AiDotNet.Reasoning.Benchmarks.Models
Represents a single problem in a benchmark dataset.
For Beginners
This is one test question with its correct answer. Think of it like a single problem on a homework assignment or test.
Example (GSM8K):
Problem: "Janet has 15 apples. She gives 40% to her friend. How many does she have left?"
CorrectAnswer: "9"
Example (HumanEval):
Problem: "Write a function that returns True if a number is prime, False otherwise"
CorrectAnswer: "def is_prime(n): ..." (reference implementation)
Constructors
| Constructor | Summary |
|---|---|
BenchmarkProblem() |
Properties
| Property | Summary |
|---|---|
Dictionary<string, object> Metadata { get; set; } |
Additional metadata specific to the benchmark. |
string Category { get; set; } |
Category or topic of this problem (e.g., “algebra”, “geometry”, “sorting”). |
string CorrectAnswer { get; set; } |
The correct answer or solution. |
string Difficulty { get; set; } |
Difficulty level (e.g., “easy”, “medium”, “hard”). |
string Id { get; set; } |
Unique identifier for this problem. |
string Problem { get; set; } |
The problem statement or question. |
Methods
| Method | Summary |
|---|---|
string ToString() |
Gets a summary string of this problem. |