FileStreamingDataLoader<T, TInput, TOutput>

Models & Types · AiDotNet.Data.Loaders

A streaming data loader that reads from files in a directory.

Inheritance: DataLoaderBase<T> → StreamingDataLoaderBase<T, TInput, TOutput> → FileStreamingDataLoader<T, TInput, TOutput>

For Beginners

If you have a folder full of images with labels in the filename or a separate label file, this loader will read them one by one as needed.

Example:

How It Works

FileStreamingDataLoader automatically discovers files in a directory and streams them during training. This is ideal for image datasets where each file is a sample.

Constructors

Constructor Summary
FileStreamingDataLoader(string directory, string filePattern, Func<string, CancellationToken, Task<ValueTuple<TInput, TOutput>>> fileProcessor, int batchSize, SearchOption searchOption, int prefetchCount, int numWorkers) Initializes a new instance of the FileStreamingDataLoader class.

Properties

Property Summary
IReadOnlyList<string> FilePaths { get; } Gets all file paths in the dataset.
int SampleCount { get; }
string Name { get; }