MultiSourceMixer<TItem>

Models & Types · AiDotNet.Data.Pipeline

Mixes multiple data sources with configurable weights for multi-domain training.

How It Works

Combines multiple datasets into a single stream with weighted sampling. Each batch is drawn from sources according to their mixing weights. Commonly used for training on mixtures of domains (e.g., code + text + math).

Constructors

Constructor Summary
MultiSourceMixer(int numSources, MultiSourceMixerOptions options) Creates a new multi-source mixer.

Properties

Property Summary
int NumSources { get; } Gets the number of sources.

Methods

Method Summary
IEnumerable<TItem> Mix(IReadOnlyList<IEnumerator<TItem>> sources, int totalItems) Mixes items from multiple sources using weighted sampling.
double GetWeight(int sourceIndex) Gets the normalized weight for a source.
int SelectSource() Selects which source to draw the next item from.