ArcDataLoaderOptions
Options & Configuration · AiDotNet.Data.Text.Benchmarks
Configuration options for the AI2 Reasoning Challenge (ARC) data loader.
How It Works
ARC (Clark et al. 2018) is a 4-way multiple-choice grade-school science QA benchmark. The Challenge subset contains questions that simple retrieval/co-occurrence baselines fail; Easy subset is simpler. Both are standard 0-shot LM-eval components.
Constructors
| Constructor | Summary |
|---|---|
ArcDataLoaderOptions() |
Properties
| Property | Summary |
|---|---|
ArcVariant Variant { get; set; } |
Which ARC variant: Easy or Challenge. |
DatasetSplit Split { get; set; } |
Dataset split to load. |
bool AutoDownload { get; set; } |
Automatically download if not present. |
int MaxSequenceLength { get; set; } |
Maximum (question + choice) sequence length in tokens. |
int VocabularySize { get; set; } |
Maximum vocabulary size. |
int? MaxSamples { get; set; } |
Optional maximum number of samples to load. |
string DataPath { get; set; } |
Root data path. |
Methods
| Method | Summary |
|---|---|
void Validate() |
Validates that all option values are within acceptable ranges. |