FileStreamingDataLoader<T, TInput, TOutput>
Models & Types · AiDotNet.Data.Loaders
A streaming data loader that reads from files in a directory.
Inheritance: DataLoaderBase<T> → StreamingDataLoaderBase<T, TInput, TOutput> → FileStreamingDataLoader<T, TInput, TOutput>
For Beginners
If you have a folder full of images with labels in the filename or a separate label file, this loader will read them one by one as needed.
Example:
How It Works
FileStreamingDataLoader automatically discovers files in a directory and streams them during training. This is ideal for image datasets where each file is a sample.
Constructors
| Constructor | Summary |
|---|---|
FileStreamingDataLoader(string directory, string filePattern, Func<string, CancellationToken, Task<ValueTuple<TInput, TOutput>>> fileProcessor, int batchSize, SearchOption searchOption, int prefetchCount, int numWorkers) |
Initializes a new instance of the FileStreamingDataLoader class. |
Properties
| Property | Summary |
|---|---|
IReadOnlyList<string> FilePaths { get; } |
Gets all file paths in the dataset. |
int SampleCount { get; } |
|
string Name { get; } |