CsvStreamingDataLoader<T, TInput, TOutput>
Models & Types · AiDotNet.Data.Loaders
A streaming data loader that reads from a CSV file line by line.
Inheritance: DataLoaderBase<T> → StreamingDataLoaderBase<T, TInput, TOutput> → CsvStreamingDataLoader<T, TInput, TOutput>
For Beginners
If you have a large CSV file (gigabytes of data), this loader will read it row by row as needed during training.
Example:
How It Works
CsvStreamingDataLoader reads a CSV file line by line without loading the entire file into memory. This is ideal for large tabular datasets.
Constructors
| Constructor | Summary |
|---|---|
CsvStreamingDataLoader(string filePath, Func<string, int, ValueTuple<TInput, TOutput>> lineParser, int batchSize, bool hasHeader, int prefetchCount, int numWorkers) |
Initializes a new instance of the CsvStreamingDataLoader class. |
Properties
| Property | Summary |
|---|---|
int SampleCount { get; } |
|
string Name { get; } |
Methods
| Method | Summary |
|---|---|
IEnumerable<ValueTuple<TInput[], TOutput[]>> GetSequentialBatches(int? batchSize, bool dropLast) |
Iterates through the CSV file sequentially without loading all lines into memory. |