InterleavedDataset<T>
Models & Types · AiDotNet.Data.Multimodal
A dataset where each sample is an interleaved sequence of modality segments, such as alternating image-text-image-text in vision-language models.
For Beginners
Think of a web page with alternating text paragraphs and images. This dataset represents each “page” as a sequence of typed segments:
How It Works
Interleaved sequences are common in modern multimodal models like GPT-4V, Gemini, and Flamingo, where images and text are interspersed in a single sequence. Each segment has a modality type and position, enabling the model to process mixed-modality input natively.
Constructors
| Constructor | Summary |
|---|---|
InterleavedDataset() |
Properties
| Property | Summary |
|---|---|
InterleavedSequence<T> Item { get; } |
|
int Count { get; } |
Gets the total number of interleaved sequences in the dataset. |
int MaxSegmentCount { get; } |
Gets the maximum number of segments across all sequences. |
Methods
| Method | Summary |
|---|---|
IEnumerable<ValueTuple<int, int, InterleavedSegment<T>>> GetSegmentsByModality(ModalityType modality) |
Extracts all segments of a given modality type across all sequences. |
void Add(InterleavedSequence<T> sequence) |
Adds an interleaved sequence to the dataset. |
void AddRange(IEnumerable<InterleavedSequence<T>> sequences) |
Adds multiple sequences to the dataset. |