ARCAGIBenchmark<T> |
ARC-AGI (Abstract Reasoning Corpus - Artificial General Intelligence) benchmark. |
BenchmarkProblem |
Represents a single problem in a benchmark dataset. |
BenchmarkResult<T> |
Results from evaluating a reasoning system on a benchmark. |
BoolQBenchmark<T> |
BoolQ benchmark for evaluating yes/no question answering. |
ChainOfThoughtStrategy<T> |
Implements Chain-of-Thought (CoT) reasoning that solves problems through explicit step-by-step thinking. |
CodeReasoner<T> |
Specialized reasoner for code-related problems including generation, debugging, and explanation. |
CodeXGlueBenchmark<T> |
CodeXGLUE benchmark harness (dataset-loader + metric computation). |
CodeXGlueProblem |
A single CodeXGLUE problem record (task-agnostic). |
CommonsenseQABenchmark<T> |
CommonsenseQA benchmark for evaluating commonsense knowledge and reasoning. |
DROPBenchmark<T> |
DROP (Discrete Reasoning Over Paragraphs) benchmark for numerical and discrete reasoning. |
GSM8KBenchmark<T> |
Grade School Math 8K (GSM8K) benchmark for evaluating mathematical reasoning. |
GSM8KDataLoader |
Loader for GSM8K benchmark dataset. |
GSM8KProblem |
|
HellaSwagBenchmark<T> |
HellaSwag benchmark for evaluating commonsense natural language inference. |
HumanEvalBenchmark<T> |
HumanEval benchmark for evaluating Python code generation capabilities. |
HumanEvalDataLoader |
Loader for HumanEval benchmark dataset. |
HumanEvalProblem |
|
LogiQABenchmark<T> |
LogiQA benchmark for evaluating logical reasoning abilities. |
LogicalReasoner<T> |
Specialized reasoner for formal logic and logical reasoning problems. |
MATHBenchmark<T> |
MATH benchmark for evaluating advanced mathematical reasoning. |
MBPPBenchmark<T> |
MBPP (Mostly Basic Python Problems) benchmark for evaluating Python code generation. |
MMLUBenchmark<T> |
MMLU (Massive Multitask Language Understanding) benchmark for evaluating world knowledge. |
MathematicalReasoner<T> |
Specialized reasoner for mathematical problems using verified reasoning and external verification. |
PIQABenchmark<T> |
PIQA (Physical Interaction Question Answering) benchmark for physical commonsense reasoning. |
ProblemEvaluation<T> |
Result for a single problem evaluation. |
ReasoningChain<T> |
Represents a complete chain of reasoning steps from problem to solution. |
ReasoningResult<T> |
Represents the complete result of a reasoning process, including the answer, reasoning chain, and performance metrics. |
ReasoningStep<T> |
Represents a single step in a reasoning chain, capturing the thought process and evaluation. |
ScientificReasoner<T> |
Specialized reasoner for scientific problems and hypotheses. |
SelfConsistencyStrategy<T> |
Implements Self-Consistency reasoning by sampling multiple reasoning paths and using majority voting. |
TreeOfThoughtsStrategy<T> |
Implements Tree-of-Thoughts (ToT) reasoning that explores multiple reasoning paths in a tree structure. |
TruthfulQABenchmark<T> |
TruthfulQA benchmark for evaluating truthfulness and resistance to falsehoods. |
WinoGrandeBenchmark<T> |
WinoGrande benchmark for evaluating commonsense reasoning through pronoun resolution. |