TinyStoriesDataLoaderOptions
Options & Configuration · AiDotNet.Data.Text.Benchmarks
Configuration options for the TinyStories data loader (Eldan & Li, 2023).
How It Works
TinyStories is a synthetic LM corpus generated by GPT-3.5/4 to teach small (10M-param) language models to produce coherent short narratives. ≈ 2.1M training stories, ≈ 22k validation stories. Used to test whether architectural choices help small LMs at all (Eldan & Li 2023, “TinyStories: How Small Can Language Models Be and Still Speak Coherent English?”). Files are plain text with each story separated by <|endoftext|>.
Constructors
| Constructor | Summary |
|---|---|
TinyStoriesDataLoaderOptions() |
Properties
| Property | Summary |
|---|---|
DatasetSplit Split { get; set; } |
Dataset split to load. |
bool AutoDownload { get; set; } |
Automatically download if not present. |
int SequenceLength { get; set; } |
Sequence length per training sample. |
int VocabularySize { get; set; } |
Maximum vocabulary size. |
int? MaxSamples { get; set; } |
Optional maximum number of samples to load. |
string DataPath { get; set; } |
Root data path. |
Methods
| Method | Summary |
|---|---|
void Validate() |
Validates that all option values are within acceptable ranges. |