DynamicBatchSampler
Models & Types ยท AiDotNet.Data.Sampling
Creates batches that fit a maximum number of tokens/elements rather than a fixed number of samples.
Inheritance: DataSamplerBase โ DynamicBatchSampler
Implements: IBatchSampler
How It Works
Unlike fixed-size batching (N samples per batch), dynamic batching fits as many samples as possible up to a token/element budget. This maximizes GPU utilization for variable-length inputs by ensuring each batch uses roughly the same amount of memory regardless of sequence lengths.
For example, with a max tokens budget of 512: a batch might contain 8 sequences of length 64, or 2 sequences of length 256, or 1 sequence of length 512.
Constructors
| Constructor | Summary |
|---|---|
DynamicBatchSampler(int[] sampleLengths, int maxTokensPerBatch, int maxSamplesPerBatch, bool shuffle, bool dropLast, int? seed) |
Creates a new dynamic batch sampler. |
Properties
| Property | Summary |
|---|---|
bool DropLast { get; set; } |
|
int BatchSize { get; set; } |
Gets or sets the maximum number of samples per batch (upper bound even if token budget allows more). |
int Length { get; } |
Methods
| Method | Summary |
|---|---|
IEnumerable<int[]> GetBatchIndices() |