MultiSourceMixer<TItem>
Models & Types · AiDotNet.Data.Pipeline
Mixes multiple data sources with configurable weights for multi-domain training.
How It Works
Combines multiple datasets into a single stream with weighted sampling. Each batch is drawn from sources according to their mixing weights. Commonly used for training on mixtures of domains (e.g., code + text + math).
Constructors
| Constructor | Summary |
|---|---|
MultiSourceMixer(int numSources, MultiSourceMixerOptions options) |
Creates a new multi-source mixer. |
Properties
| Property | Summary |
|---|---|
int NumSources { get; } |
Gets the number of sources. |
Methods
| Method | Summary |
|---|---|
IEnumerable<TItem> Mix(IReadOnlyList<IEnumerator<TItem>> sources, int totalItems) |
Mixes items from multiple sources using weighted sampling. |
double GetWeight(int sourceIndex) |
Gets the normalized weight for a source. |
int SelectSource() |
Selects which source to draw the next item from. |