IGroundedConversationTrainable<T>

Interfaces · AiDotNet.VisionLanguage.Interfaces

A model that trains on grounded conversations: an image, a conversation about it, the regions its questions name, and a mask for every [SEG] in its answers.

Known implementations: GLaMM<T>

How It Works

Such a model’s loss combines the language-model loss with mask losses; a tensor target cannot carry the conversation, so these models refuse Train(input, expected) and train here instead.

Methods

Method Summary
T EvaluateConversationLoss(GlammSample<T> sample) The paper loss on the same sample, without training.
void TrainConversation(GlammSample<T> sample) One training step with the model’s paper loss.