LeafRedditFederatedDatasetLoader
Models & Types · AiDotNet.FederatedLearning.Benchmarks.Leaf
Loads the LEAF Reddit benchmark JSON files into per-client token-sequence datasets.
For Beginners
Reddit is huge. This loader supports loading a subset of users and sampling per user so you can run CI-friendly benchmark checks.
How It Works
The LEAF Reddit preprocessing pipeline stores each sample as a list of token chunks (x) and a metadata
object (y) containing target_tokens (shifted next-token targets) and optional count_tokens.
This loader converts each sample into a single fixed-length token sequence paired with a single next-token label
(v1: last non-pad target token).
Constructors
| Constructor | Summary |
|---|---|
LeafRedditFederatedDatasetLoader() |
Methods
| Method | Summary |
|---|---|
LeafFederatedDataset<string[][], string[]> LoadDatasetFromFiles(string trainFilePath, string testFilePath, LeafFederatedDatasetLoadOptions options) |
Loads a LEAF Reddit train dataset and optional test dataset from files. |
LeafFederatedSplit<string[][], string[]> LoadSplitFromFile(string filePath, LeafFederatedDatasetLoadOptions options) |
Loads a LEAF Reddit split (train/test) from a JSON file. |
LeafFederatedSplit<string[][], string[]> LoadSplitFromJson(string json, LeafFederatedDatasetLoadOptions options) |
Loads a LEAF Reddit split (train/test) from a JSON string. |