InputOutputDataLoaderBase<T, TInput, TOutput>

Base Classes ยท AiDotNet.Data.Loaders

Abstract base class for input-output data loaders providing common supervised learning functionality.

Inheritance: DataLoaderBase<T> โ†’ InputOutputDataLoaderBase<T, TInput, TOutput>

Implements: IInputOutputDataLoader<T, TInput, TOutput>

Known derived types: Ade20kDataLoader<T>, AgNewsDataLoader<T>, ArcDataLoader<T>, AudioFileDataset<T>, AudioSetDataLoader<T>, BigEarthNetDataLoader<T>, Caltech101DataLoader<T>, CelebADataLoader<T>, CheXpertDataLoader<T>, ChestXray14DataLoader<T>, Cifar100DataLoader<T>, Cifar10DataLoader<T>, CityscapesDataLoader<T>, CnnDailyMailDataLoader<T>, CocoDetectionDataLoader<T>, CommonVoiceDataLoader<T>, CsvDataLoader<T>, DocVqaDataLoader<T>, DtdDataLoader<T>, Enwik8DataLoader<T>, Esc50DataLoader<T>, EuroSatDataLoader<T>, FMoWDataLoader<T>, FashionMnistDataLoader<T>, FinancialDataLoader<T>, FleursDataLoader<T>, Flowers102DataLoader<T>, Food101DataLoader<T>, Fsd50kDataLoader<T>, GigaSpeechDataLoader<T>, GlueDataLoader<T>, Gsm8kDataLoader<T>, GtzanDataLoader<T>, HellaswagDataLoader<T>, Hmdb51DataLoader<T>, HumanEvalDataLoader<T>, INaturalistDataLoader<T>, ImageClassificationDataset<T>, ImageFolderDataset<T>, ImageNet1kDataLoader<T>, ImageNet21kDataLoader<T>, Imdb50kDataLoader<T>, InMemoryDataLoader<T, TInput, TOutput>, Kinetics400DataLoader<T>, KittiDataLoader<T>, LeafFederatedDataLoader<T>, LibriSpeechDataLoader<T>, LjSpeechDataLoader<T>, MaestroDataLoader<T>, MathDataLoader<T>, MbppDataLoader<T>, MmluDataLoader<T>, MnistDataLoader<T>, Musdb18DataLoader<T>, NsynthDataLoader<T>, NuScenesDataLoader<T>, OpenImagesDataLoader<T>, OxfordPetsDataLoader<T>, ParquetDataLoader<T>, PascalVocDataLoader<T> (+32 more)

For Beginners

This base class handles common input-output data operations:

  • Storing features (X) and labels (Y) for supervised learning
  • Splitting data into training, validation, and test sets
  • Shuffling data to improve training
  • Iterating through data in batches

Concrete implementations (CsvDataLoader, ImageDataLoader) extend this to load specific data formats.

How It Works

InputOutputDataLoaderBase provides shared implementation for all supervised learning data loaders including:

  • Feature (X) and label (Y) data management
  • Train/validation/test splitting
  • Shuffling and batching capabilities
  • Progress tracking through ICountable

Properties

Property Summary
TInput Features { get; }
TOutput Labels { get; }
bool HasNext { get; }
bool IsShuffled { get; }
int BatchSize { get; set; }
int FeatureCount { get; }
int OutputDimension { get; }

Methods

Method Summary
IAsyncEnumerable<ValueTuple<TInput, TOutput>> GetBatchesAsync(int? batchSize, bool shuffle, bool dropLast, int? seed, int prefetchCount, CancellationToken cancellationToken)
IEnumerable<ValueTuple<TInput, TOutput>> GetBatches(int? batchSize, bool shuffle, bool dropLast, int? seed)
ValueTuple<IInputOutputDataLoader<T, TInput, TOutput>, IInputOutputDataLoader<T, TInput, TOutput>, IInputOutputDataLoader<T, TInput, TOutput>> Split(double trainRatio, double validationRatio, int? seed)
ValueTuple<TInput, TOutput> GetNextBatch()
bool TryGetNextBatch(ValueTuple<TInput, TOutput> batch)
void Shuffle(int? seed)
void Unshuffle()