BenchmarkSuite

Enums · AiDotNet.Enums

Defines the supported benchmark suites available through the AiDotNet facade.

For Beginners

A benchmark suite is like a standardized test you can run to measure how well your model performs on a specific family of problems. You select a suite using this enum, and AiDotNet runs the benchmark and returns a structured report.

Fields

Field Summary
BenchmarkSuite ARCAGI ARC-AGI - abstract reasoning puzzles.
BenchmarkSuite BoolQ BoolQ - yes/no question answering.
BenchmarkSuite CIFAR10 CIFAR-10 - federated image classification (synthetic partitioning of standard CIFAR-10).
BenchmarkSuite CIFAR100 CIFAR-100 - federated image classification (synthetic partitioning of standard CIFAR-100).
BenchmarkSuite CommonsenseQA CommonsenseQA - commonsense multiple-choice QA.
BenchmarkSuite DROP DROP - reading comprehension with discrete reasoning over paragraphs.
BenchmarkSuite FEMNIST FEMNIST - LEAF federated handwritten character classification (per-writer partitioning).
BenchmarkSuite GSM8K Grade School Math 8K (GSM8K) - multi-step math word problems.
BenchmarkSuite HellaSwag HellaSwag - commonsense inference in narrative completion.
BenchmarkSuite HumanEval HumanEval - code generation / program synthesis evaluation.
BenchmarkSuite LAMBADA LAMBADA - full-context open-vocabulary final-word prediction over narrative passages.
BenchmarkSuite LEAF LEAF - federated benchmark suite (JSON-based train/test splits).
BenchmarkSuite LogiQA LogiQA - logical reasoning benchmark.
BenchmarkSuite MATH MATH - competition-style mathematics problems.
BenchmarkSuite MBPP MBPP - mostly basic programming problems.
BenchmarkSuite MMLU MMLU - broad multi-subject multiple-choice benchmark.
BenchmarkSuite PIQA PIQA - physical commonsense reasoning.
BenchmarkSuite Perplexity Perplexity - non-overlapping causal language-modeling cross-entropy and perplexity evaluation.
BenchmarkSuite Reddit Reddit - federated next-token prediction benchmark (LEAF Reddit dataset).
BenchmarkSuite Sent140 Sent140 - LEAF federated sentiment classification benchmark based on tweets.
BenchmarkSuite Shakespeare Shakespeare - LEAF federated next-character prediction benchmark.
BenchmarkSuite StackOverflow StackOverflow - federated next-token prediction benchmark (StackOverflow corpus).
BenchmarkSuite TabularNonIID Generic tabular suite with synthetic non-IID client partitions.
BenchmarkSuite TruthfulQA TruthfulQA - evaluates truthfulness and resistance to hallucination.
BenchmarkSuite WinoGrande WinoGrande - pronoun resolution / commonsense reasoning.
int value__