Gsm8kDataLoader<T>

Models & Types · AiDotNet.Data.Text.Benchmarks

Loads the GSM8K grade-school math word-problem dataset (Cobbe et al. 2021).

Inheritance: DataLoaderBase<T> → InputOutputDataLoaderBase<T, Tensor<T>, Tensor<T>> → Gsm8kDataLoader<T>

How It Works

Expects:

Each line is {"question": "...", "answer": "... #### N"}. Auto-download fetches the canonical OpenAI release. Features are tokenized question sequences Tensor[N, MaxQuestionLength]; labels are tokenized answer (chain-of-thought + final number) sequences Tensor[N, MaxAnswerLength].

Constructors

Constructor Summary
Gsm8kDataLoader(Gsm8kDataLoaderOptions options) Creates a new GSM8K data loader.

Properties

Property Summary
int FeatureCount { get; }
int OutputDimension { get; }
int TotalCount { get; }
string Description { get; }
string Name { get; }

Methods

Method Summary
ValueTuple<IInputOutputDataLoader<T, Tensor<T>, Tensor<T>>, IInputOutputDataLoader<T, Tensor<T>, Tensor<T>>, IInputOutputDataLoader<T, Tensor<T>, Tensor<T>>> Split(double trainRatio, double validationRatio, int? seed)