IVisionLanguageAction<T>

Interfaces · AiDotNet.VisionLanguage.Interfaces

Interface for Vision-Language-Action (VLA) models that connect visual understanding and language reasoning to physical robotic actions.

Implements: IGenerativeVisionLanguageModel<T>

Known implementations: GR00TN1<T>, Helix<T>, Octo<T>, PaLME<T>, PiZero<T>, RT2<T>, ThreeDVLA<T>

How It Works

VLA models bridge the gap between perception (vision-language understanding) and action (robotic control). They take visual observations and optional language instructions to predict action sequences for robot manipulation, navigation, and planning.

Properties

Property Summary
int ActionDimension { get; } Gets the dimensionality of the action space (e.g., number of joint DOFs).
string LanguageModelName { get; } Gets the name of the language model backbone.

Methods

Method Summary
Tensor<T> PredictAction(Tensor<T> observation, string instruction) Predicts an action sequence from a visual observation and a language instruction.