HumanEvalDataLoaderOptions
Options & Configuration · AiDotNet.Data.Text.Benchmarks
Configuration options for the HumanEval Python code-generation benchmark (Chen et al. 2021).
How It Works
HumanEval is a 164-problem hand-curated Python function-completion benchmark. Each problem provides a function signature + docstring as the prompt; the model completes the body. Pass@k scores against the canonical unit tests are the standard metric. Used as the default code-generation benchmark since GPT-3.5/Codex.
Constructors
| Constructor | Summary |
|---|---|
HumanEvalDataLoaderOptions() |
Properties
| Property | Summary |
|---|---|
DatasetSplit Split { get; set; } |
Dataset split to load. |
bool AutoDownload { get; set; } |
Automatically download if not present. |
int MaxPromptLength { get; set; } |
Maximum prompt length in tokens. |
int MaxSolutionLength { get; set; } |
Maximum solution length in tokens. |
int VocabularySize { get; set; } |
Maximum vocabulary size. |
int? MaxSamples { get; set; } |
Optional maximum number of samples to load. |
string DataPath { get; set; } |
Root data path. |
Methods
| Method | Summary |
|---|---|
void Validate() |
Validates that all option values are within acceptable ranges. |