InputOutputDataLoaderBase<T, TInput, TOutput>
Base Classes ยท AiDotNet.Data.Loaders
Abstract base class for input-output data loaders providing common supervised learning functionality.
Inheritance: DataLoaderBase<T> โ InputOutputDataLoaderBase<T, TInput, TOutput>
Implements: IInputOutputDataLoader<T, TInput, TOutput>
Known derived types: Ade20kDataLoader<T>, AgNewsDataLoader<T>, ArcDataLoader<T>, AudioFileDataset<T>, AudioSetDataLoader<T>, BigEarthNetDataLoader<T>, Caltech101DataLoader<T>, CelebADataLoader<T>, CheXpertDataLoader<T>, ChestXray14DataLoader<T>, Cifar100DataLoader<T>, Cifar10DataLoader<T>, CityscapesDataLoader<T>, CnnDailyMailDataLoader<T>, CocoDetectionDataLoader<T>, CommonVoiceDataLoader<T>, CsvDataLoader<T>, DocVqaDataLoader<T>, DtdDataLoader<T>, Enwik8DataLoader<T>, Esc50DataLoader<T>, EuroSatDataLoader<T>, FMoWDataLoader<T>, FashionMnistDataLoader<T>, FinancialDataLoader<T>, FleursDataLoader<T>, Flowers102DataLoader<T>, Food101DataLoader<T>, Fsd50kDataLoader<T>, GigaSpeechDataLoader<T>, GlueDataLoader<T>, Gsm8kDataLoader<T>, GtzanDataLoader<T>, HellaswagDataLoader<T>, Hmdb51DataLoader<T>, HumanEvalDataLoader<T>, INaturalistDataLoader<T>, ImageClassificationDataset<T>, ImageFolderDataset<T>, ImageNet1kDataLoader<T>, ImageNet21kDataLoader<T>, Imdb50kDataLoader<T>, InMemoryDataLoader<T, TInput, TOutput>, Kinetics400DataLoader<T>, KittiDataLoader<T>, LeafFederatedDataLoader<T>, LibriSpeechDataLoader<T>, LjSpeechDataLoader<T>, MaestroDataLoader<T>, MathDataLoader<T>, MbppDataLoader<T>, MmluDataLoader<T>, MnistDataLoader<T>, Musdb18DataLoader<T>, NsynthDataLoader<T>, NuScenesDataLoader<T>, OpenImagesDataLoader<T>, OxfordPetsDataLoader<T>, ParquetDataLoader<T>, PascalVocDataLoader<T> (+32 more)
For Beginners
This base class handles common input-output data operations:
- Storing features (X) and labels (Y) for supervised learning
- Splitting data into training, validation, and test sets
- Shuffling data to improve training
- Iterating through data in batches
Concrete implementations (CsvDataLoader, ImageDataLoader) extend this to load specific data formats.
How It Works
InputOutputDataLoaderBase provides shared implementation for all supervised learning data loaders including:
- Feature (X) and label (Y) data management
- Train/validation/test splitting
- Shuffling and batching capabilities
- Progress tracking through ICountable
Properties
| Property | Summary |
|---|---|
TInput Features { get; } |
|
TOutput Labels { get; } |
|
bool HasNext { get; } |
|
bool IsShuffled { get; } |
|
int BatchSize { get; set; } |
|
int FeatureCount { get; } |
|
int OutputDimension { get; } |
Methods
| Method | Summary |
|---|---|
IAsyncEnumerable<ValueTuple<TInput, TOutput>> GetBatchesAsync(int? batchSize, bool shuffle, bool dropLast, int? seed, int prefetchCount, CancellationToken cancellationToken) |
|
IEnumerable<ValueTuple<TInput, TOutput>> GetBatches(int? batchSize, bool shuffle, bool dropLast, int? seed) |
|
ValueTuple<IInputOutputDataLoader<T, TInput, TOutput>, IInputOutputDataLoader<T, TInput, TOutput>, IInputOutputDataLoader<T, TInput, TOutput>> Split(double trainRatio, double validationRatio, int? seed) |
|
ValueTuple<TInput, TOutput> GetNextBatch() |
|
bool TryGetNextBatch(ValueTuple<TInput, TOutput> batch) |
|
void Shuffle(int? seed) |
|
void Unshuffle() |