Data

349 public types in this namespace, organized by kind.

Models & Types (191)

Type Summary
ActiveLearningQueryStrategy Selects the most informative unlabeled samples for annotation using active learning.
ActiveLearningSampler<T> A sampler for active learning that selects the most informative samples for labeling.
Ade20kDataLoader<T> Loads the ADE20K semantic segmentation dataset.
AgNewsDataLoader<T> Loads the AG News topic classification dataset (4 classes: World, Sports, Business, Sci/Tech).
ArcDataLoader<T> Loads the AI2 Reasoning Challenge (ARC) multiple-choice science QA benchmark (Clark et al.
ArrowDataset<T> Provides read/write access to datasets in a custom binary columnar format inspired by Apache Arrow IPC.
AsyncDataPipeline<T> Provides async data pipeline operations with prefetching support.
AudioFileDataset<T> Loads audio files from directories for audio classification and processing tasks.
AudioSetDataLoader<T> Loads the AudioSet large-scale audio event dataset (2M+ 10-second clips, 527 categories).
BalancedEpisodicDataLoader<T, TInput, TOutput> Provides balanced episodic task sampling that ensures equal class representation across multiple tasks.
BigEarthNetDataLoader<T> Loads the BigEarthNet multi-label remote sensing dataset (590K Sentinel-2 patches, 19 or 43 classes).
BucketBatchSampler Groups sequences by length into buckets, then batches within each bucket to minimize padding.
CacheInfo Information about the current state of a pipeline cache.
CachingDataLoader<TKey, TValue> Wraps data loading with an in-memory cache to avoid redundant I/O.
Caltech101DataLoader<T> Loads the Caltech-101 image classification dataset (Fei-Fei et al.
CelebADataLoader<T> Loads the CelebA face attributes dataset.
CheXpertDataLoader<T> Loads the CheXpert chest radiograph dataset (224K images, 14 observations with uncertainty).
CheckpointData Data stored in a mid-epoch checkpoint.
ChestXray14DataLoader<T> Loads the NIH Chest X-ray 14 multi-label classification dataset (112K images, 14 disease labels).
Cifar100DataLoader<T> Loads the CIFAR-100 image classification dataset (50k train / 10k test, 32x32 RGB, 100 fine / 20 coarse classes).
Cifar10DataLoader<T> Loads the CIFAR-10 image classification dataset (50k train / 10k test, 32x32 RGB, 10 classes).
CitationNetworkLoader<T> Loads citation network datasets (Cora, CiteSeer, PubMed) for node classification.
CityscapesDataLoader<T> Loads the Cityscapes semantic-segmentation dataset (Cordts et al.
CnnDailyMailDataLoader<T> Loads the CNN/DailyMail abstractive-summarization dataset v3.0.0 (Hermann et al.
CocoDetectionDataLoader<T> Loads the COCO 2017 object detection dataset (118K train / 5K val, 80 categories).
CommonVoiceDataLoader<T> Loads the Mozilla Common Voice multilingual speech dataset (19K+ hours, 100+ languages).
Compose<T> Chains multiple transforms of the same type into a single transform.
CoresetSelector Selects a representative coreset from a dataset using distance-based strategies.
CsvDataLoader<T> Loads supervised learning data from CSV files into Matrix/Vector format.
CsvStreamingDataLoader<T, TInput, TOutput> A streaming data loader that reads from a CSV file line by line.
CurriculumDataScheduler Schedules training data presentation order and pacing for curriculum learning.
CurriculumEpisodicDataLoader<T, TInput, TOutput> Provides curriculum-based episodic task sampling that progressively increases task difficulty during training.
CurriculumSampler<T> A sampler that implements curriculum learning by progressively introducing harder samples.
DataPipeline<T> Provides TensorFlow-style data pipeline operations for transforming and processing data.
DataPruner Prunes (removes) training samples based on difficulty/importance scores.
DatasetDistiller Performs dataset distillation to synthesize a compact representative dataset.
DatasetMixer<T> Blends multiple datasets by weight ratios, producing mixed batches for curriculum or domain mixing.
DefaultCollateFunction<T> Stacks equal-size tensors into a batch tensor along dimension 0.
DenseSampler Dense sampling: uniformly samples frames at regular intervals from the video.
DistributedBucketSampler Combines distributed partitioning with bucket batching for efficient distributed NLP training.
DistributedSampler Partitions dataset indices across N ranks for distributed (multi-GPU/multi-node) training.
DistributedStreamingDataLoader<T, TInput, TOutput> Wraps any IStreamingDataLoader to partition each global batch deterministically across distributed ranks, supporting both homogeneous and heterogeneous per-rank micro-batch sizes as well as exact sample-offset fast-forwarding across world…
DocVqaDataLoader<T> Loads the DocVQA document visual question answering dataset.
DomainMixingSampler Samples from multiple data domains with configurable mixing ratios for multi-domain LLM training.
DtdDataLoader<T> Loads the Describable Textures Dataset (DTD; Cimpoi et al.
DynamicBatchSampler Creates batches that fit a maximum number of tokens/elements rather than a fixed number of samples.
ElasticDistributedSampler Distributed sampler that evenly divides data across multiple workers with elastic scaling support.
EnvironmentDataLoader<T> Data loader for reinforcement learning that wraps an environment for experience collection.
Enwik8DataLoader<T> Loads the enwik8 character-level Wikipedia language modeling benchmark (first 100M bytes of an English Wikipedia XML dump).
Esc50DataLoader<T> Loads the ESC-50 environmental sound classification dataset (2000 clips, 50 classes).
EuroSatDataLoader<T> Loads the EuroSAT land use/land cover classification dataset (27K patches, 64x64 RGB, 10 classes).
ExactHashDeduplicator Detects exact duplicate documents using cryptographic hashing (SHA-256).
FMoWDataLoader<T> Loads the Functional Map of the World (fMoW) satellite imagery dataset (1M+ images, 62 categories).
FashionMnistDataLoader<T> Loads the Fashion-MNIST clothing classification dataset (60k train / 10k test, 28x28 grayscale).
FileStreamingDataLoader<T, TInput, TOutput> A streaming data loader that reads from files in a directory.
FleursDataLoader<T> Loads the FLEURS multilingual speech benchmark (102 languages, ~12 hours per language).
Flowers102DataLoader<T> Loads the Oxford Flowers-102 dataset (Nilsback & Zisserman 2008) — 102 fine-grained flower species.
Food101DataLoader<T> Loads the Food-101 fine-grained classification dataset (Bossard et al.
Fsd50kDataLoader<T> Loads the FSD50K audio event dataset (51,197 clips, 200 sound event classes).
GigaSpeechDataLoader<T> Loads the GigaSpeech multi-domain English ASR dataset (10K hours from audiobooks, podcasts, YouTube).
GlueDataLoader<T> Loads GLUE benchmark sub-tasks (CoLA, SST-2, MRPC, QQP, STS-B, MNLI, QNLI, RTE, WNLI).
GraphClassificationTask<T> Represents a graph classification task where the goal is to classify entire graphs.
GraphData<T> Represents a single graph with nodes, edges, features, and optional labels.
GraphGenerationTask<T> Represents a graph generation task where the goal is to generate new valid graphs.
Gsm8kDataLoader<T> Loads the GSM8K grade-school math word-problem dataset (Cobbe et al.
GtzanDataLoader<T> Loads the GTZAN music genre classification dataset (Tzanetakis & Cook 2002).
Hdf5Dataset<T> Provides read/write access to datasets in a custom binary format for named multidimensional arrays, inspired by the HDF5 data model.
HellaswagDataLoader<T> Loads the HellaSwag 4-way commonsense NLI benchmark (Zellers et al.
HeuristicTextFilter Filters text documents using simple heuristic rules for quality assessment.
Hmdb51DataLoader<T> Loads the HMDB51 action recognition dataset (6,766 clips, 51 classes).
HumanEvalDataLoader<T> Loads the HumanEval Python code-generation benchmark (Chen et al.
INaturalistDataLoader<T> Loads the iNaturalist species classification dataset (~2.7M images, 10,000 species in 2021 version).
IdentityTransform<T> A no-op transform that passes input through unchanged.
ImageClassificationDataset<T> An in-memory image classification dataset with an optional composable transform pipeline.
ImageFolderDataset<T> Loads images from a directory structure where each subdirectory is a class label.
ImageNet1kDataLoader<T> Loads the ImageNet-1K (ILSVRC 2012) image classification dataset (~1.28M train / 50K val, 1000 classes).
ImageNet21kDataLoader<T> Loads the ImageNet-21K dataset (~14.2M images, 21,841 categories from the full WordNet hierarchy).
ImageQualityFilter Filters images based on resolution, aspect ratio, and pixel statistics.
Imdb50kDataLoader<T> Loads the IMDB 50k movie review sentiment analysis dataset (25k train / 25k test, binary classification).
ImportanceSampler<T> A sampler that implements importance sampling for variance reduction.
InMemoryDataLoader<T, TInput, TOutput> A simple in-memory data loader for supervised learning data.
InterleavedDataset<T> A dataset where each sample is an interleaved sequence of modality segments, such as alternating image-text-image-text in vision-language models.
InterleavedSegment<T> A single segment within an interleaved sequence.
InterleavedSequence<T> A single interleaved sequence containing ordered segments of different modalities.
JsonlDataLoader<T> A typed data loader facade that wraps JsonlStreamingLoader and implements StreamingDataLoaderBase for IDataLoader compliance.
Kinetics400DataLoader<T> Loads the Kinetics-400 human action recognition dataset (~300K clips, 400 classes).
KittiDataLoader<T> Loads the KITTI 3D object detection dataset (LiDAR point clouds with 3D bounding boxes).
LambdaTransform<TInput, TOutput> Wraps a Func delegate as an ITransform.
LanguageIdFilter Filters documents based on detected language using character n-gram profiles.
LeafFederatedDataLoader<T> Data loader that reads LEAF benchmark JSON splits and exposes both aggregated (X, Y) data and per-client partitions.
LibriSpeechDataLoader<T> Loads the LibriSpeech automatic speech recognition dataset (~1000 hours of 16kHz English speech).
LinkPredictionTask<T> Represents a link prediction task where the goal is to predict missing or future edges.
LjSpeechDataLoader<T> Loads the LJSpeech 1.1 single-speaker TTS corpus (Ito & Johnson 2017).
LmdbDataset<T> Provides read-only access to datasets stored in a custom binary key-value format inspired by LMDB (Lightning Memory-Mapped Database).
M4DatasetLoader<T> Loads time series datasets from the M4 Competition for benchmarking forecasting models.
M4TimeSeries<T> Represents a single time series from the M4 Competition.
MaestroDataLoader<T> Loads the MAESTRO piano performance dataset (~200 hours, aligned MIDI and audio).
MathDataLoader<T> Loads the Hendrycks MATH benchmark — competition math problems (Hendrycks et al.
MbppDataLoader<T> Loads the MBPP (Mostly Basic Python Problems) benchmark — 1,000 entry-level Python coding problems with natural-language descriptions and unit tests (Austin et al.
MemoryMappedDataset<T> Memory-mapped dataset access for efficient I/O on large binary datasets.
MemoryMappedStreamingDataLoader<T, TInput, TOutput> A streaming data loader that uses memory-mapped files for efficient random access to large binary datasets.
MetaLearningTask<T, TInput, TOutput> Represents a single meta-learning task for few-shot learning, containing support and query sets.
MidEpochCheckpointer Saves and restores training state mid-epoch for fault tolerance.
MinHashDeduplicator Detects near-duplicate documents using MinHash with Locality-Sensitive Hashing (LSH).
MinMaxScaleTransform<T> Scales values to a target range using min-max normalization.
MmluDataLoader<T> Loads MMLU — Massive Multitask Language Understanding (Hendrycks et al.
MnistDataLoader<T> Loads the MNIST handwritten digit classification dataset (60k train / 10k test, 28x28 grayscale).
ModalitySample<T> Represents a single modality’s data within a multimodal sample.
ModelNet40ClassificationDataLoader<T> Loads the ModelNet40 point cloud classification dataset.
MolecularDatasetLoader<T> Loads molecular graph datasets (ZINC, QM9) for graph-level property prediction and generation.
MultiSourceMixer<TItem> Mixes multiple data sources with configurable weights for multi-domain training.
MultimodalDataset<T> A dataset of multimodal samples for training models that process multiple data types simultaneously.
MultimodalSample<T> A collection of modality samples representing one multimodal data point.
Musdb18DataLoader<T> Loads the MUSDB18 music source separation dataset (150 tracks, 4 stems: vocals, drums, bass, other).
NodeClassificationTask<T> Represents a node classification task where the goal is to predict labels for individual nodes.
NormalizeTransform<T> Normalizes an array of values using mean and standard deviation: (x - mean) / std.
NsynthDataLoader<T> Loads the NSynth Neural Synth dataset (Engel et al.
NuScenesDataLoader<T> Loads the nuScenes dataset (LiDAR point clouds with 3D bounding box annotations).
OGBDatasetLoader<T> Loads datasets from the Open Graph Benchmark (OGB) for standardized evaluation.
OneHotEncodeTransform<T> Converts a class index (integer label) to a one-hot encoded vector.
OpenImagesDataLoader<T> Loads the Open Images V7 object detection dataset (~9M images, 600 categories).
OxfordPetsDataLoader<T> Loads the Oxford-IIIT Pet dataset (Parkhi et al.
PackedSequenceBatch<T> Represents a batch of packed (non-padded) variable-length sequences.
PackedSequenceCollateFunction<T> Packs variable-length sequences into a contiguous tensor without padding, along with sequence lengths for reconstruction.
PaddingCollateFunction<T> Pads variable-length sequences to the maximum length in the batch, then stacks them.
ParallelBatchLoader<TBatch> Provides parallel batch loading with multiple workers for improved throughput.
ParquetDataLoader<T> Reads tabular data from Apache Parquet columnar files using the Parquet.Net library.
PascalVocDataLoader<T> Loads the Pascal VOC object detection dataset (20 categories, XML annotations).
PennTreebankDataLoader<T> Loads the Penn Treebank language modeling dataset (Mikolov-preprocessed split).
PerplexityFilter Filters documents based on perplexity scores from a simple n-gram language model.
Places365DataLoader<T> Loads the Places365 scene recognition dataset (1.8M train / 36.5K val, 256x256 RGB, 365 scene categories).
PrefetchDataLoader<TBatch> Wraps a batch-producing function with asynchronous prefetching for pipelined data loading.
ProteinDataLoader<T> Loads protein structure datasets as flattened feature/label tensors for graph-level classification.
PubLayNetDataLoader<T> Loads the PubLayNet document layout analysis dataset.
Qm9DataLoader<T> Thin wrapper around MolecularDatasetLoader for the QM9 dataset.
RandomSampler A sampler that randomly shuffles the dataset indices each epoch.
RetinalFundusDataLoader<T> Loads retinal fundus photography datasets for diabetic retinopathy detection (5-class grading).
ScanNetSemanticSegmentationDataLoader<T> Loads the ScanNet semantic segmentation dataset.
SelfPacedSampler<T> A sampler that implements self-paced learning with automatic difficulty adjustment.
SemanticDeduplicator Detects semantic duplicates using embedding cosine similarity.
SemanticKittiDataLoader<T> Loads the SemanticKITTI dataset (per-point semantic labels for LiDAR point clouds).
SequencePackingCollateFunction<T> Packs multiple variable-length sequences into fixed-length blocks for efficient LLM training.
SequentialSampler A sampler that returns indices in sequential order without shuffling.
ShapeNetCorePartSegmentationDataLoader<T> Loads the ShapeNetCore part segmentation dataset.
ShardedStreamingDataLoader<T> A typed data loader facade that wraps ShardedStreamingDataset and implements StreamingDataLoaderBase for IDataLoader compliance.
SkinLesionDataLoader<T> Loads the ISIC Skin Lesion classification dataset (~25K images, 8 diagnostic categories).
SlowFastSampler SlowFast sampling: produces two sets of frame indices at different temporal resolutions for SlowFast networks (Feichtenhofer et al., 2019).
SnapshotPipeline<T> Persists an entire processed pipeline to disk for fast reload across epochs, with automatic invalidation when source data or pipeline configuration changes.
SomethingSomethingV2DataLoader<T> Loads the Something-Something V2 action recognition dataset (220K clips, 174 classes).
SpecAugmentTransform<T> Applies SpecAugment data augmentation (Park et al., 2019) to spectrogram tensors.
SpectrogramTransform<T> Transforms raw audio waveform tensors into Mel spectrogram representations.
SpeechCommandsDataLoader<T> Loads the Google Speech Commands v2 dataset (~65K clips, 35 words, 16kHz).
SquadDataLoader<T> Loads the SQuAD question answering dataset (100K+ Q&A pairs on Wikipedia articles).
StandardScaleTransform<T> Applies Z-score normalization: (x - mean) / std, computing mean and std from a reference dataset.
StanfordCarsDataLoader<T> Loads the Stanford Cars dataset (Krause et al.
StatefulDataLoader<T, TInput, TOutput> Wraps any InMemoryDataLoader with checkpoint/resume support.
Stl10DataLoader<T> Loads the STL-10 image classification dataset (Coates et al.
StratifiedBatchSampler A batch sampler that ensures each batch contains samples from all classes.
StratifiedEpisodicDataLoader<T, TInput, TOutput> Provides stratified episodic task sampling that maintains dataset class proportions across tasks.
StratifiedSampler A sampler that ensures each class is represented proportionally in each epoch.
StreamingDataLoader<T, TInput, TOutput> A data loader that streams data from disk or other sources without loading all data into memory.
StreamingTextDataset<T> A streaming text dataset that lazily reads and tokenizes text files for language model training.
SubsetSampler A sampler that returns a subset of indices.
SuperGlueDataLoader<T> Loads SuperGLUE benchmark sub-tasks (BoolQ, CB, COPA, MultiRC, ReCoRD, RTE, WiC, WSC).
SvhnDataLoader<T> Loads the SVHN (Street View House Numbers) Format-2 dataset (Netzer et al.
TaskBatch<T, TInput, TOutput> Represents a batch of tasks for meta-learning with advanced batching strategies.
TemporalGraphDataLoader<T> Loads temporal graph datasets (timestamped interactions for dynamic link prediction).
TemporalJitterAugmentation<T> Applies temporal jitter to video frame sequences by randomly shifting frame indices.
TemporalSegmentSampler Temporal Segment Networks (TSN) sampling: divides video into equal segments and samples one frame per segment.
TextLineDataset<T> Streams text line-by-line from a file for language modeling tasks.
TimitDataLoader<T> Loads the TIMIT acoustic-phonetic continuous-speech corpus (Garofolo et al.
TinyImageNetDataLoader<T> Loads the Tiny ImageNet 200-class image-classification dataset (500 train + 50 val + 50 test per class at 64×64).
TinyStoriesDataLoader<T> Loads the TinyStories synthetic LM corpus (Eldan & Li, 2023): ≈ 2.1M GPT-generated short stories with deliberately small vocabulary for small-scale language-model research.
ToTensorTransform<T> Converts a flat array to a Tensor with the specified shape.
TokenizedTextDataset<T> In-memory dataset of pre-tokenized text sequences for language model training.
TransformedDataLoader<T> Wraps a data loader and applies a composable transform pipeline to feature data during batch extraction.
TruthfulQaDataLoader<T> Loads the TruthfulQA benchmark (Lin et al.
Ucf101DataLoader<T> Loads the UCF101 action recognition dataset (13,320 clips, 101 classes).
UniformEpisodicDataLoader<T, TInput, TOutput> Provides uniform random episodic task sampling for N-way K-shot meta-learning scenarios.
UrbanSound8kDataLoader<T> Loads the UrbanSound8K environmental sound dataset (8732 clips, 10 classes, 10 folds).
VctkDataLoader<T> Loads the VCTK Corpus 0.92 multi-speaker TTS dataset.
VideoFrameDataset<T> Loads videos represented as directories of sequentially numbered image frames.
VoxPopuliDataLoader<T> Loads the VoxPopuli multilingual speech dataset from European Parliament recordings.
WaymoDataLoader<T> Loads the Waymo Open Dataset (LiDAR point clouds with 3D bounding boxes).
WebDatasetDataLoader<T> A typed data loader facade that wraps WebDataset and implements StreamingDataLoaderBase for IDataLoader compliance.
WeightedSampler<T> A sampler that samples indices based on their weights.
WikiText103DataLoader<T> Loads the WikiText-103 language modeling dataset (100M+ tokens from Wikipedia articles).
WikiText2DataLoader<T> Loads the WikiText-2 language modeling dataset (≈ 2M tokens train, ≈ 245k val, ≈ 281k test from Wikipedia articles).
Wikidata5mDataLoader<T> Loads Wikidata5M knowledge graph triplets as tensor features and labels.
XSumDataLoader<T> Loads the XSum extreme abstractive-summarization dataset (Narayan et al.
ZincDataLoader<T> Thin wrapper around MolecularDatasetLoader for the ZINC dataset.

Base Classes (11)

Type Summary
DataLoaderBase<T> Abstract base class providing common functionality for all data loaders.
DataSamplerBase Base class for all data samplers providing common functionality.
EpisodicDataLoaderBase<T, TInput, TOutput> Provides a base implementation for episodic data loaders with common functionality for N-way K-shot meta-learning.
EpochAdaptiveSamplerBase<T> Base class for epoch-adaptive samplers that change behavior over training epochs.
GraphDataLoaderBase<T> Abstract base class for graph data loaders providing common graph-related functionality.
InputOutputDataLoaderBase<T, TInput, TOutput> Abstract base class for input-output data loaders providing common supervised learning functionality.
MetaLearningTaskBase<T, TInput, TOutput> Abstract base class for meta-learning tasks, providing common functionality and validation.
PointCloudDatasetLoaderBase<T> Base class for point cloud dataset loaders that expose tensor inputs and outputs.
RLDataLoaderBase<T> Abstract base class for RL data loaders providing common reinforcement learning functionality.
StreamingDataLoaderBase<T, TInput, TOutput> Abstract base class for streaming data loaders that process data on-demand.
WeightedSamplerBase<T> Base class for weighted samplers providing common weight-based functionality.

Interfaces (3)

Type Summary
ICollateFunction<TSample, TBatch> Defines how individual samples are assembled into a batch.
ITransform<TInput, TOutput> Core interface for composable data transforms in the data pipeline.
IVideoFrameSampler Interface for video frame sampling strategies.

Enums (26)

Type Summary
ActiveLearningStrategy Active learning selection strategies.
ArcVariant Variant of the ARC benchmark to load.
BatchingStrategy Defines strategies for batching tasks in meta-learning.
CacheEvictionPolicy Policy for evicting cache entries when the cache is full.
CitationDataset<T> Available citation network datasets.
CoresetStrategy Strategy for selecting coreset samples.
CurriculumOrder Order in which curriculum samples are presented.
CurriculumPacing Pacing function controlling how fast the data pool grows.
CurriculumStage Represents stages in a meta-learning curriculum.
CurriculumStrategy Defines how the curriculum progresses over epochs.
DatasetSplit Standard dataset split options.
GlueTask GLUE benchmark sub-tasks.
ImageTensorLayout Specifies the axis ordering for image tensors returned by vision data loaders.
M4Frequency Frequency categories in the M4 Competition.
MemoryCacheEvictionPolicy Policy for evicting items from the in-memory cache when it’s full.
ModalityType Identifies the type of data modality in a multimodal sample.
MolecularDataset<T> Available molecular datasets.
OGBTask<T> OGB task types.
PointPaddingStrategy Strategies for padding point clouds when fewer points than requested exist.
PointSamplingStrategy Strategies for sampling points from a point cloud.
PruneStrategy Strategy for selecting samples to prune.
QueryStrategy Strategy for selecting samples in active learning.
ScanNetInputFormat Supported ScanNet input data formats.
ScanNetLabelMode ScanNet label mapping modes.
SuperGlueTask SuperGLUE benchmark sub-tasks.
UncertaintyPolicy Policy for handling uncertain labels in CheXpert.

Options & Configuration (114)

Type Summary
ActiveLearningQueryStrategyOptions Configuration options for active learning query strategies.
Ade20kDataLoaderOptions Configuration options for the ADE20K semantic segmentation data loader.
AgNewsDataLoaderOptions Configuration options for the AG News topic classification data loader.
ArcDataLoaderOptions Configuration options for the AI2 Reasoning Challenge (ARC) data loader.
ArrowDatasetOptions Configuration options for Apache Arrow-based dataset access.
AudioFileDatasetOptions Configuration options for the AudioFileDataset.
AudioSetDataLoaderOptions Configuration options for the AudioSet data loader.
BigEarthNetDataLoaderOptions Configuration options for the BigEarthNet data loader.
CachingDataLoaderOptions Configuration options for caching data loader.
Caltech101DataLoaderOptions Configuration options for the Caltech-101 image classification data loader (Fei-Fei et al.
CelebADataLoaderOptions Configuration options for the CelebA face attributes data loader.
CheXpertDataLoaderOptions Configuration options for the CheXpert data loader.
ChestXray14DataLoaderOptions Configuration options for the NIH Chest X-ray 14 data loader.
Cifar100DataLoaderOptions Configuration options for the CIFAR-100 data loader.
Cifar10DataLoaderOptions Configuration options for the CIFAR-10 data loader.
CityscapesDataLoaderOptions Configuration options for the Cityscapes semantic segmentation loader.
CnnDailyMailDataLoaderOptions Configuration options for the CNN/DailyMail summarization loader (Hermann et al.
CocoDetectionDataLoaderOptions Configuration options for the COCO Detection data loader.
CommonVoiceDataLoaderOptions Configuration options for the Mozilla Common Voice data loader.
CoresetSelectorOptions Configuration options for coreset selection.
CurriculumDataSchedulerOptions Configuration options for curriculum data scheduling.
DataPrunerOptions Configuration options for data pruning.
DatasetDistillerOptions Configuration options for dataset distillation.
DiskCacheOptions Configuration options for disk-based pipeline caching.
DocVqaDataLoaderOptions Configuration options for the DocVQA (Document Visual Question Answering) data loader.
DtdDataLoaderOptions Configuration options for the Describable Textures Dataset (DTD) loader (Cimpoi et al.
ElasticDistributedSamplerOptions Configuration options for elastic distributed sampling.
Enwik8DataLoaderOptions Configuration options for the enwik8 character-level LM data loader.
Esc50DataLoaderOptions Configuration options for the ESC-50 data loader.
EuroSatDataLoaderOptions Configuration options for the EuroSAT data loader.
ExactHashDeduplicatorOptions Configuration options for exact hash-based deduplication.
FMoWDataLoaderOptions Configuration options for the Functional Map of the World (fMoW) data loader.
FashionMnistDataLoaderOptions Configuration options for the Fashion-MNIST data loader.
FleursDataLoaderOptions Configuration options for the FLEURS data loader.
Flowers102DataLoaderOptions Configuration options for the Oxford Flowers-102 dataset (Nilsback & Zisserman 2008).
Food101DataLoaderOptions Configuration options for the Food-101 image classification data loader.
Fsd50kDataLoaderOptions Configuration options for the FSD50K data loader.
GigaSpeechDataLoaderOptions Configuration options for the GigaSpeech data loader.
GlueDataLoaderOptions Configuration options for the GLUE benchmark data loader.
Gsm8kDataLoaderOptions Configuration options for the GSM8K math word-problem benchmark.
GtzanDataLoaderOptions Configuration options for the GTZAN music genre classification loader.
Hdf5DatasetOptions Configuration options for HDF5 dataset access.
HellaswagDataLoaderOptions Configuration options for the HellaSwag commonsense NLI benchmark (Zellers et al.
HeuristicTextFilterOptions Configuration options for heuristic text quality filtering.
Hmdb51DataLoaderOptions Configuration options for the HMDB51 data loader.
HumanEvalDataLoaderOptions Configuration options for the HumanEval Python code-generation benchmark (Chen et al.
INaturalistDataLoaderOptions Configuration options for the iNaturalist data loader.
ImageFolderDatasetOptions Configuration options for the ImageFolderDataset.
ImageNet1kDataLoaderOptions Configuration options for the ImageNet-1K (ILSVRC 2012) data loader.
ImageNet21kDataLoaderOptions Configuration options for the ImageNet-21K data loader.
ImageQualityFilterOptions Configuration options for image quality filtering.
Imdb50kDataLoaderOptions Configuration options for the IMDB 50k sentiment analysis data loader.
Kinetics400DataLoaderOptions Configuration options for the Kinetics-400 data loader.
KittiDataLoaderOptions Configuration options for the KITTI 3D object detection data loader.
LanguageIdFilterOptions Configuration options for language identification filtering.
LibriSpeechDataLoaderOptions Configuration options for the LibriSpeech data loader.
LjSpeechDataLoaderOptions Configuration options for the LJSpeech 1.1 data loader (Ito & Johnson 2017).
LmdbDatasetOptions Configuration options for LMDB-based dataset access.
MaestroDataLoaderOptions Configuration options for the MAESTRO data loader.
MathDataLoaderOptions Configuration options for the Hendrycks MATH benchmark loader.
MbppDataLoaderOptions Configuration options for the Mostly Basic Python Problems (MBPP) loader (Austin et al.
MidEpochCheckpointerOptions Configuration options for mid-epoch checkpointing.
MinHashDeduplicatorOptions Configuration options for MinHash-based near-duplicate detection.
MmluDataLoaderOptions Configuration options for the MMLU (Massive Multitask Language Understanding) loader (Hendrycks et al.
MnistDataLoaderOptions Configuration options for the MNIST data loader.
ModelNet40ClassificationDataLoaderOptions Configuration options for the ModelNet40 classification data loader.
MultiSourceMixerOptions Configuration options for multi-source data mixing.
Musdb18DataLoaderOptions Configuration options for the MUSDB18 data loader.
NsynthDataLoaderOptions Configuration for the NSynth (Neural Synth) audio dataset loader (Engel et al.
NuScenesDataLoaderOptions Configuration options for the nuScenes data loader.
OpenImagesDataLoaderOptions Configuration options for the Open Images V7 data loader.
OxfordPetsDataLoaderOptions Configuration options for the Oxford-IIIT Pet dataset loader (Parkhi et al.
ParallelBatchLoaderConfig Configuration for parallel batch loading.
ParquetDataLoaderOptions Configuration options for the ParquetDataLoader.
PascalVocDataLoaderOptions Configuration options for the Pascal VOC data loader.
PennTreebankDataLoaderOptions Configuration options for the Penn Treebank (PTB) data loader.
PerplexityFilterOptions Configuration options for perplexity-based text quality filtering.
Places365DataLoaderOptions Configuration options for the Places365 data loader.
PrefetchDataLoaderOptions Configuration options for prefetch-enabled data loading.
ProteinDataLoaderOptions Configuration options for the protein structure graph data loader.
PubLayNetDataLoaderOptions Configuration options for the PubLayNet document layout analysis data loader.
Qm9DataLoaderOptions Configuration options for the QM9 molecular property prediction data loader.
RetinalFundusDataLoaderOptions Configuration options for the Retinal Fundus data loader.
ScanNetSemanticSegmentationDataLoaderOptions Configuration options for the ScanNet semantic segmentation data loader.
SemanticDeduplicatorOptions Configuration options for semantic-level deduplication using embeddings.
SemanticKittiDataLoaderOptions Configuration options for the SemanticKITTI data loader.
ShapeNetCorePartSegmentationDataLoaderOptions Configuration options for the ShapeNetCore part segmentation data loader.
ShardedStreamingDatasetOptions Configuration options for the sharded streaming dataset.
SkinLesionDataLoaderOptions Configuration options for the ISIC Skin Lesion data loader.
SomethingSomethingV2DataLoaderOptions Configuration options for the Something-Something V2 data loader.
SpeechCommandsDataLoaderOptions Configuration options for the Google Speech Commands v2 data loader.
SquadDataLoaderOptions Configuration options for the SQuAD data loader.
StanfordCarsDataLoaderOptions Configuration for the Stanford Cars dataset loader (Krause et al.
Stl10DataLoaderOptions Configuration options for the STL-10 image classification dataset (Coates et al.
StreamingTextDatasetOptions Configuration options for the streaming text dataset.
SuperGlueDataLoaderOptions Configuration options for the SuperGLUE benchmark data loader.
SvhnDataLoaderOptions Configuration options for the SVHN (Street View House Numbers) loader (Netzer et al.
TemporalGraphDataLoaderOptions Configuration options for the temporal graph data loader.
TimitDataLoaderOptions Configuration for the TIMIT acoustic-phonetic continuous-speech corpus loader (Garofolo et al.
TinyImageNetDataLoaderOptions Configuration options for the Tiny ImageNet (200-class, 64×64) data loader.
TinyStoriesDataLoaderOptions Configuration options for the TinyStories data loader (Eldan & Li, 2023).
TruthfulQaDataLoaderOptions Configuration for the TruthfulQA benchmark loader (Lin et al.
Ucf101DataLoaderOptions Configuration options for the UCF101 data loader.
UrbanSound8kDataLoaderOptions Configuration options for the UrbanSound8K data loader.
VctkDataLoaderOptions Configuration for the VCTK Corpus 0.92 multi-speaker TTS loader (Yamagishi et al.
VideoFrameDatasetOptions Configuration options for the VideoFrameDataset.
VoxPopuliDataLoaderOptions Configuration options for the VoxPopuli data loader.
WaymoDataLoaderOptions Configuration options for the Waymo Open Dataset data loader.
WebDatasetOptions Configuration options for the WebDataset loader.
WikiText103DataLoaderOptions Configuration options for the WikiText-103 data loader.
WikiText2DataLoaderOptions Configuration options for the WikiText-2 data loader.
Wikidata5mDataLoaderOptions Configuration options for the Wikidata5M knowledge graph data loader.
XSumDataLoaderOptions Configuration options for the XSum extreme summarization loader (Narayan et al.
ZincDataLoaderOptions Configuration options for the ZINC molecular dataset data loader.

Helpers & Utilities (4)

Type Summary
DataLoaders Static factory class for creating data loaders with beginner-friendly methods.
DataPipelineExtensions Extension methods for creating data pipelines from various sources.
ParallelBatchLoaderExtensions Provides extension methods for parallel batch loading.
Samplers Static factory class for creating data samplers with beginner-friendly methods.