MmluDataLoader<T>
Models & Types · AiDotNet.Data.Text.Benchmarks
Loads MMLU — Massive Multitask Language Understanding (Hendrycks et al. 2021).
Inheritance: DataLoaderBase<T> → InputOutputDataLoaderBase<T, Tensor<T>, Tensor<T>> → MmluDataLoader<T>
How It Works
Expects the canonical AI release layout:
Each CSV has columns question, A, B, C, D, answer with no header.
Auto-downloads the canonical Hendrycks tarball. 4-way multiple choice
shape: features [N, 4, MaxQuestionLength], one-hot labels [N, 4].
Constructors
| Constructor | Summary |
|---|---|
MmluDataLoader(MmluDataLoaderOptions options) |
Properties
| Property | Summary |
|---|---|
int FeatureCount { get; } |
|
int OutputDimension { get; } |
|
int TotalCount { get; } |
|
string Description { get; } |
|
string Name { get; } |
Methods
| Method | Summary |
|---|---|
ValueTuple<IInputOutputDataLoader<T, Tensor<T>, Tensor<T>>, IInputOutputDataLoader<T, Tensor<T>, Tensor<T>>, IInputOutputDataLoader<T, Tensor<T>, Tensor<T>>> Split(double trainRatio, double validationRatio, int? seed) |