HumanEvalDataLoaderOptions

Options & Configuration · AiDotNet.Data.Text.Benchmarks

Configuration options for the HumanEval Python code-generation benchmark (Chen et al. 2021).

How It Works

HumanEval is a 164-problem hand-curated Python function-completion benchmark. Each problem provides a function signature + docstring as the prompt; the model completes the body. Pass@k scores against the canonical unit tests are the standard metric. Used as the default code-generation benchmark since GPT-3.5/Codex.

Constructors

Constructor Summary
HumanEvalDataLoaderOptions()

Properties

Property Summary
DatasetSplit Split { get; set; } Dataset split to load.
bool AutoDownload { get; set; } Automatically download if not present.
int MaxPromptLength { get; set; } Maximum prompt length in tokens.
int MaxSolutionLength { get; set; } Maximum solution length in tokens.
int VocabularySize { get; set; } Maximum vocabulary size.
int? MaxSamples { get; set; } Optional maximum number of samples to load.
string DataPath { get; set; } Root data path.

Methods

Method Summary
void Validate() Validates that all option values are within acceptable ranges.