BenchmarkSuite
Enums · AiDotNet.Enums
Defines the supported benchmark suites available through the AiDotNet facade.
For Beginners
A benchmark suite is like a standardized test you can run to measure how well your model performs on a specific family of problems. You select a suite using this enum, and AiDotNet runs the benchmark and returns a structured report.
Fields
| Field | Summary |
|---|---|
BenchmarkSuite ARCAGI |
ARC-AGI - abstract reasoning puzzles. |
BenchmarkSuite BoolQ |
BoolQ - yes/no question answering. |
BenchmarkSuite CIFAR10 |
CIFAR-10 - federated image classification (synthetic partitioning of standard CIFAR-10). |
BenchmarkSuite CIFAR100 |
CIFAR-100 - federated image classification (synthetic partitioning of standard CIFAR-100). |
BenchmarkSuite CommonsenseQA |
CommonsenseQA - commonsense multiple-choice QA. |
BenchmarkSuite DROP |
DROP - reading comprehension with discrete reasoning over paragraphs. |
BenchmarkSuite FEMNIST |
FEMNIST - LEAF federated handwritten character classification (per-writer partitioning). |
BenchmarkSuite GSM8K |
Grade School Math 8K (GSM8K) - multi-step math word problems. |
BenchmarkSuite HellaSwag |
HellaSwag - commonsense inference in narrative completion. |
BenchmarkSuite HumanEval |
HumanEval - code generation / program synthesis evaluation. |
BenchmarkSuite LAMBADA |
LAMBADA - full-context open-vocabulary final-word prediction over narrative passages. |
BenchmarkSuite LEAF |
LEAF - federated benchmark suite (JSON-based train/test splits). |
BenchmarkSuite LogiQA |
LogiQA - logical reasoning benchmark. |
BenchmarkSuite MATH |
MATH - competition-style mathematics problems. |
BenchmarkSuite MBPP |
MBPP - mostly basic programming problems. |
BenchmarkSuite MMLU |
MMLU - broad multi-subject multiple-choice benchmark. |
BenchmarkSuite PIQA |
PIQA - physical commonsense reasoning. |
BenchmarkSuite Perplexity |
Perplexity - non-overlapping causal language-modeling cross-entropy and perplexity evaluation. |
BenchmarkSuite Reddit |
Reddit - federated next-token prediction benchmark (LEAF Reddit dataset). |
BenchmarkSuite Sent140 |
Sent140 - LEAF federated sentiment classification benchmark based on tweets. |
BenchmarkSuite Shakespeare |
Shakespeare - LEAF federated next-character prediction benchmark. |
BenchmarkSuite StackOverflow |
StackOverflow - federated next-token prediction benchmark (StackOverflow corpus). |
BenchmarkSuite TabularNonIID |
Generic tabular suite with synthetic non-IID client partitions. |
BenchmarkSuite TruthfulQA |
TruthfulQA - evaluates truthfulness and resistance to hallucination. |
BenchmarkSuite WinoGrande |
WinoGrande - pronoun resolution / commonsense reasoning. |
int value__ |