TruthfulQaDataLoaderOptions
Options & Configuration ยท AiDotNet.Data.Text.Benchmarks
Configuration for the TruthfulQA benchmark loader (Lin et al. 2022).
How It Works
TruthfulQA tests whether language models avoid common false beliefs in 38 categories โ 817 questions, each with a best correct answer and multiple correct/incorrect distractors. This loader exposes the generation-style version (best_answer as target).
Constructors
| Constructor | Summary |
|---|---|
TruthfulQaDataLoaderOptions() |
Properties
| Property | Summary |
|---|---|
DatasetSplit Split { get; set; } |
Gets or sets which split of the dataset is loaded. |
bool AutoDownload { get; set; } |
Gets or sets whether the dataset is downloaded into the data directory when it is not there. |
int MaxAnswerLength { get; set; } |
Gets or sets the token length each answer is padded or truncated to. |
int MaxQuestionLength { get; set; } |
Gets or sets the token length each question is padded or truncated to. |
int VocabularySize { get; set; } |
Gets or sets the most distinct tokens in the vocabulary built from the loaded text. |
int? MaxSamples { get; set; } |
Gets or sets the most samples loaded; null loads the whole split. |
string DataPath { get; set; } |
Gets or sets the dataset root directory; null uses the default data directory for this dataset. |
Methods
| Method | Summary |
|---|---|
void Validate() |
Validates that all option values are within acceptable ranges. |