ThreeDVLA<T>
Models & Types · AiDotNet.VisionLanguage.Robotics
3D-VLA: connects vision-language-action to 3D world via generative world model.
Inheritance: NeuralNetworkBase<T> → VisionLanguageModelBase<T> → ThreeDVLA<T>
Implements: IVisionLanguageAction<T>
For Beginners
3D-VLA is a vision-language-action model that uses 3D world modeling for robot planning and action generation. Default values follow the original paper settings.
How It Works
3D-VLA (UMass, 2024) connects vision-language-action models to the 3D world via a generative world model. It processes 3D point cloud observations alongside language instructions and generates both predicted future states (as a world model) and robot actions, enabling planning through imagination in 3D space.
References:
- Paper: “3D-VLA: A 3D Vision-Language-Action Generative World Model (UMass, 2024)”
Example
// Create a 3D-VLA model for 3D vision-language-action reasoning
// connecting VLA to 3D world via a generative world model
var architecture = new NeuralNetworkArchitecture<double>(
inputType: InputType.TwoDimensional,
taskType: NeuralNetworkTaskType.ImageClassification,
inputHeight: 224, inputWidth: 224, inputDepth: 3, outputSize: 512);
// ONNX inference mode with pre-trained model
var model = new ThreeDVLA<double>(architecture, "threedvla.onnx");
// Training mode with native layers
var trainModel = new ThreeDVLA<double>(architecture, new ThreeDVLAOptions());
Constructors
| Constructor | Summary |
|---|---|
ThreeDVLA(NeuralNetworkArchitecture<T> architecture, ThreeDVLAOptions options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer) |
|
ThreeDVLA(NeuralNetworkArchitecture<T> architecture, string modelPath, ThreeDVLAOptions options) |
Properties
| Property | Summary |
|---|---|
int ActionDimension { get; } |
|
int DecoderEmbeddingDim { get; } |
|
int EmbeddingDimension { get; } |
|
int MaxGenerationLength { get; } |
|
string LanguageModelName { get; } |
Methods
| Method | Summary |
|---|---|
ModelMetadata<T> GetModelMetadata() |
|
Tensor<T> EncodeImage(Tensor<T> image) |
|
Tensor<T> GenerateFromImage(Tensor<T> image, string prompt) |
Generates from image using 3D-VLA’s generative world model approach. |
Tensor<T> PredictAction(Tensor<T> observation, string instruction) |
Predicts action using 3D-VLA’s generative world model approach. |