InterleavedDataset<T>

Models & Types · AiDotNet.Data.Multimodal

A dataset where each sample is an interleaved sequence of modality segments, such as alternating image-text-image-text in vision-language models.

For Beginners

Think of a web page with alternating text paragraphs and images. This dataset represents each “page” as a sequence of typed segments:

How It Works

Interleaved sequences are common in modern multimodal models like GPT-4V, Gemini, and Flamingo, where images and text are interspersed in a single sequence. Each segment has a modality type and position, enabling the model to process mixed-modality input natively.

Constructors

Constructor Summary
InterleavedDataset()

Properties

Property Summary
InterleavedSequence<T> Item { get; }
int Count { get; } Gets the total number of interleaved sequences in the dataset.
int MaxSegmentCount { get; } Gets the maximum number of segments across all sequences.

Methods

Method Summary
IEnumerable<ValueTuple<int, int, InterleavedSegment<T>>> GetSegmentsByModality(ModalityType modality) Extracts all segments of a given modality type across all sequences.
void Add(InterleavedSequence<T> sequence) Adds an interleaved sequence to the dataset.
void AddRange(IEnumerable<InterleavedSequence<T>> sequences) Adds multiple sequences to the dataset.