DataPipelineExtensions

Helpers & Utilities · AiDotNet.Data.Pipeline

Extension methods for creating data pipelines from various sources.

Methods

Method Summary
AsyncDataPipeline<ValueTuple<TInput[], TOutput[]>> ToAsyncPipeline(IStreamingDataLoader<T, TInput, TOutput> loader, bool shuffle, int? seed) Creates an async DataPipeline from a streaming data loader.
DataPipeline<T> Sample(DataPipeline<T> source, IReadOnlyList<double> weights, int numSamples, int? seed) Samples elements with replacement using the given weights.
DataPipeline<T> ToPipeline(IEnumerable<T> source) Creates a DataPipeline from an enumerable.
DataPipeline<TBatch> ToPipeline(IBatchIterable<TBatch> source) Creates a DataPipeline from an enumerable.
DataPipeline<T[]> PaddedBatch(DataPipeline<T> source, int batchSize, T padValue) Creates batches with padding to ensure uniform batch sizes.
DataPipeline<T[]> Window(DataPipeline<T> source, int windowSize, int? shift) Applies window-based operations to the pipeline.
DataPipeline<ValueTuple<TInput, TOutput>> ToSamplePipeline(IStreamingDataLoader<T, TInput, TOutput> loader, bool shuffle, int? seed) Creates a DataPipeline of individual samples from a streaming data loader.
DataPipeline<ValueTuple<TInput[], TOutput[]>> ToPipeline(IStreamingDataLoader<T, TInput, TOutput> loader, bool shuffle, int? seed) Creates a DataPipeline from a streaming data loader.