DynamicBatchSampler

Models & Types ยท AiDotNet.Data.Sampling

Creates batches that fit a maximum number of tokens/elements rather than a fixed number of samples.

Inheritance: DataSamplerBase โ†’ DynamicBatchSampler

Implements: IBatchSampler

How It Works

Unlike fixed-size batching (N samples per batch), dynamic batching fits as many samples as possible up to a token/element budget. This maximizes GPU utilization for variable-length inputs by ensuring each batch uses roughly the same amount of memory regardless of sequence lengths.

For example, with a max tokens budget of 512: a batch might contain 8 sequences of length 64, or 2 sequences of length 256, or 1 sequence of length 512.

Constructors

Constructor Summary
DynamicBatchSampler(int[] sampleLengths, int maxTokensPerBatch, int maxSamplesPerBatch, bool shuffle, bool dropLast, int? seed) Creates a new dynamic batch sampler.

Properties

Property Summary
bool DropLast { get; set; }
int BatchSize { get; set; } Gets or sets the maximum number of samples per batch (upper bound even if token budget allows more).
int Length { get; }

Methods

Method Summary
IEnumerable<int[]> GetBatchIndices()