MmluDataLoader<T>

Models & Types · AiDotNet.Data.Text.Benchmarks

Loads MMLU — Massive Multitask Language Understanding (Hendrycks et al. 2021).

Inheritance: DataLoaderBase<T> → InputOutputDataLoaderBase<T, Tensor<T>, Tensor<T>> → MmluDataLoader<T>

How It Works

Expects the canonical AI release layout:

Each CSV has columns question, A, B, C, D, answer with no header. Auto-downloads the canonical Hendrycks tarball. 4-way multiple choice shape: features [N, 4, MaxQuestionLength], one-hot labels [N, 4].

Constructors

Constructor Summary
MmluDataLoader(MmluDataLoaderOptions options)

Properties

Property Summary
int FeatureCount { get; }
int OutputDimension { get; }
int TotalCount { get; }
string Description { get; }
string Name { get; }

Methods

Method Summary
ValueTuple<IInputOutputDataLoader<T, Tensor<T>, Tensor<T>>, IInputOutputDataLoader<T, Tensor<T>, Tensor<T>>, IInputOutputDataLoader<T, Tensor<T>, Tensor<T>>> Split(double trainRatio, double validationRatio, int? seed)