LeafTokenSequenceFederatedDatasetLoader
Models & Types · AiDotNet.FederatedLearning.Benchmarks.Leaf
Loads LEAF-style JSON files that store token sequences (x) and next-token labels (y).
For Beginners
Some federated text benchmarks are easiest to represent as “predict the next token”. This loader keeps the per-user splits intact so federated simulations match the benchmark partitioning.
How It Works
This loader is intentionally generic: it expects the standard LEAF container shape
(users/num_samples/user_data) where each user’s x is a list of token sequences
(arrays of strings) and each y is a list of label tokens (strings).
Constructors
| Constructor | Summary |
|---|---|
LeafTokenSequenceFederatedDatasetLoader() |
Methods
| Method | Summary |
|---|---|
LeafFederatedDataset<string[][], string[]> LoadDatasetFromFiles(string trainFilePath, string testFilePath, LeafFederatedDatasetLoadOptions options) |
Loads a LEAF token-sequence train dataset and optional test dataset from files. |
LeafFederatedSplit<string[][], string[]> LoadSplitFromFile(string filePath, LeafFederatedDatasetLoadOptions options) |
Loads a LEAF token-sequence split (train/test) from a JSON file. |
LeafFederatedSplit<string[][], string[]> LoadSplitFromJson(string json, LeafFederatedDatasetLoadOptions options) |
Loads a LEAF token-sequence split (train/test) from a JSON string. |