JsonlDataLoader<T>

Models & Types · AiDotNet.Data.Formats

A typed data loader facade that wraps JsonlStreamingLoader and implements StreamingDataLoaderBase for IDataLoader compliance.

Inheritance: DataLoaderBase<T> → StreamingDataLoaderBase<T, Tensor<T>, Tensor<T>> → JsonlDataLoader<T>

For Beginners

Use this when you want to load JSONL files and feed them into AiModelBuilder.ConfigureDataLoader(). You provide a parser function that knows how to convert each JSON record into tensors for training.

How It Works

This loader eagerly reads all JSON objects from the underlying JSONL files during loading, caches them in memory, and provides index-based access via a user-supplied parser delegate that converts JObject records into typed tensor pairs.

Constructors

Constructor Summary
JsonlDataLoader(string[] filePaths, Func<JObject, ValueTuple<Tensor<T>, Tensor<T>>> recordParser, int batchSize, string textField, string labelField, int shuffleBufferSize) Creates a new JsonlDataLoader.

Properties

Property Summary
int SampleCount { get; }
string Name { get; }