TinyStoriesDataLoaderOptions

Options & Configuration · AiDotNet.Data.Text.Benchmarks

Configuration options for the TinyStories data loader (Eldan & Li, 2023).

How It Works

TinyStories is a synthetic LM corpus generated by GPT-3.5/4 to teach small (10M-param) language models to produce coherent short narratives. ≈ 2.1M training stories, ≈ 22k validation stories. Used to test whether architectural choices help small LMs at all (Eldan & Li 2023, “TinyStories: How Small Can Language Models Be and Still Speak Coherent English?”). Files are plain text with each story separated by <|endoftext|>.

Constructors

Constructor Summary
TinyStoriesDataLoaderOptions()

Properties

Property Summary
DatasetSplit Split { get; set; } Dataset split to load.
bool AutoDownload { get; set; } Automatically download if not present.
int SequenceLength { get; set; } Sequence length per training sample.
int VocabularySize { get; set; } Maximum vocabulary size.
int? MaxSamples { get; set; } Optional maximum number of samples to load.
string DataPath { get; set; } Root data path.

Methods

Method Summary
void Validate() Validates that all option values are within acceptable ranges.