DatasetMixer<T>

Models & Types · AiDotNet.Data.Multimodal

Blends multiple datasets by weight ratios, producing mixed batches for curriculum or domain mixing.

For Beginners

Imagine you have three types of training data: books, code, and conversations. You want your model to see 50% books, 30% code, and 20% conversations. DatasetMixer handles this:

How It Works

Dataset mixing is a crucial technique for training large language models and multimodal models. It allows combining data from multiple sources (domains) with controlled ratios, enabling:

  • Domain balancing (upweight rare domains)
  • Curriculum learning (gradually shift domain mix during training)
  • Multi-task training (combine datasets for different tasks)

Constructors

Constructor Summary
DatasetMixer()

Properties

Property Summary
IReadOnlyList<double> NormalizedWeights { get; } Gets the normalized mixing weights (summing to 1.0).
int SourceCount { get; } Gets the number of data sources in the mixer.
int TotalSamples { get; } Gets the total number of samples across all sources.

Methods

Method Summary
IEnumerable<ValueTuple<int, int>> GetMixedBatchIndices(int batchSize, int? seed) Generates batch indices following the mixing weights distribution.
IEnumerable<ValueTuple<int, int>[]> GetMixedBatches(int batchSize, int? seed) Generates an infinite stream of mixed batch indices, following mixing weights.
MultimodalSample<T> GetSample(int sourceIndex, int sampleIndex) Gets a sample from a specific source at a specific index.
ValueTuple<string, int, double>[] GetSourceInfo() Gets information about all sources.
void AddSource(MultimodalDataset<T> dataset, double weight, string name) Adds a dataset source with a mixing weight.
void AddTensorSource(Tensor<T> features, Tensor<T> labels, ModalityType modalityType, double weight, string name) Adds a tensor-based data source with a mixing weight.
void UpdateWeight(int sourceIndex, double newWeight) Updates the mixing weight for a specific source (for curriculum learning).