MultimodalSample<T>

Models & Types · AiDotNet.Data.Multimodal

A collection of modality samples representing one multimodal data point.

For Beginners

This is like a folder containing different types of data that all describe the same thing. For example:

How It Works

A MultimodalSample groups multiple modalities together as a single training example. For instance, an image-text pair for captioning would contain an Image modality and a Text modality.

Constructors

Constructor Summary
MultimodalSample(IEnumerable<ModalitySample<T>> modalities) Creates a new multimodal sample from a list of modality samples.
MultimodalSample(ModalitySample<T>[] modalities) Creates a new multimodal sample from a list of modality samples.

Properties

Property Summary
IReadOnlyCollection<string> Keys { get; } Gets the available keys for accessing modalities.
IReadOnlyList<ModalitySample<T>> Modalities { get; } Gets the modality samples in this multimodal sample.
ModalitySample<T> Item { get; }
Tensor<T> Label { get; set; } Gets optional label information for this sample.
int ClassIndex { get; set; } Gets or sets an optional integer class index.
int ModalityCount { get; } Gets the number of modalities in this sample.
string LabelName { get; set; } Gets or sets an optional string label (for classification).

Methods

Method Summary
IEnumerable<ModalitySample<T>> GetByType(ModalityType modality) Gets all modality samples of a specific type.
bool HasKey(string key) Checks whether a modality key exists in this sample.
bool HasModality(ModalityType modality) Checks whether this sample contains any modality of the specified type.
bool TryGetModality(string key, ModalitySample<T> sample) Tries to get a modality sample by key.