BenchmarkProblem

Models & Types · AiDotNet.Reasoning.Benchmarks.Models

Represents a single problem in a benchmark dataset.

For Beginners

This is one test question with its correct answer. Think of it like a single problem on a homework assignment or test.

Example (GSM8K):

Problem: "Janet has 15 apples. She gives 40% to her friend. How many does she have left?"
CorrectAnswer: "9"

Example (HumanEval):

Problem: "Write a function that returns True if a number is prime, False otherwise"
CorrectAnswer: "def is_prime(n): ..." (reference implementation)

Constructors

Constructor Summary
BenchmarkProblem()

Properties

Property Summary
Dictionary<string, object> Metadata { get; set; } Additional metadata specific to the benchmark.
string Category { get; set; } Category or topic of this problem (e.g., “algebra”, “geometry”, “sorting”).
string CorrectAnswer { get; set; } The correct answer or solution.
string Difficulty { get; set; } Difficulty level (e.g., “easy”, “medium”, “hard”).
string Id { get; set; } Unique identifier for this problem.
string Problem { get; set; } The problem statement or question.

Methods

Method Summary
string ToString() Gets a summary string of this problem.