Helix<T>
Models & Types · AiDotNet.VisionLanguage.Robotics
Helix: dual-system vision-language-action model for full-body humanoid control (Figure AI, 2025, arXiv:2502.07092).
Inheritance: NeuralNetworkBase<T> → VisionLanguageModelBase<T> → Helix<T>
Implements: IVisionLanguageAction<T>
For Beginners
Helix is the first robot brain that’s good enough to drive a real humanoid through dexterous tasks like loading a dishwasher. The trick is two networks: a smart-but- slow one decides intent (“pick up the red mug”) and a fast-but-narrow one figures out the exact motor commands needed to do it 200 times per second.
How It Works
Helix is the first generalist VLA model deployed on a real humanoid for whole upper-body dexterous manipulation. Its defining architectural choice is the dual-system, dual-rate split between a slow VLM reasoner and a fast visuomotor policy:
Paper-faithful pieces implemented here:
What is NOT verified in-session:
Example
var image = Tensor<double>.CreateRandom(1, 3, 32, 32);
var arch = new NeuralNetworkArchitecture<double>(
inputType: InputType.TwoDimensional,
taskType: NeuralNetworkTaskType.ImageClassification,
inputHeight: 224, inputWidth: 224, inputDepth: 3, outputSize: 35);
var helix = new Helix<double>(arch, new HelixOptions());
// One-shot: predict a 16-tick horizon of joint commands.
var actions = helix.PredictAction(image, "stack the cups");
// Streaming: 200 Hz control loop with caller-owned timing.
var runner = helix.CreateDualSystemRunner();
var currentInstruction = "stack the cups";
for (int tick = 0; tick < 200; tick++)
{
var currentImage = image; // your latest camera frame
var action = runner.Step(currentImage, currentInstruction);
// send `action` to the robot
}
Constructors
| Constructor | Summary |
|---|---|
Helix(NeuralNetworkArchitecture<T> architecture, HelixOptions options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer) |
|
Helix(NeuralNetworkArchitecture<T> architecture, string modelPath, HelixOptions options) |
Properties
| Property | Summary |
|---|---|
int ActionDimension { get; } |
|
int DecoderEmbeddingDim { get; } |
|
int EmbeddingDimension { get; } |
|
int MaxGenerationLength { get; } |
|
int System1ToSystem2Ratio { get; } |
Default ratio of S1 to S2 invocations (paper §4.1: 200 Hz S1, 7–9 Hz S2 → ~22:1). |
string LanguageModelName { get; } |
Methods
| Method | Summary |
|---|---|
HelixDualSystemRunner<T> CreateDualSystemRunner() |
Creates a HelixDualSystemRunner wired to this model’s S1/S2 callbacks for streaming-control use. |
ModelMetadata<T> GetModelMetadata() |
|
Tensor<T> EncodeImage(Tensor<T> image) |
|
Tensor<T> GenerateFromImage(Tensor<T> image, string prompt) |
|
Tensor<T> PredictAction(Tensor<T> observation, string instruction) |
Predicts ActionDimension × PredictionHorizon continuous joint commands using Helix’s dual-system inference path: one S2 invocation produces the semantic latent, then S1 rolls out PredictionHorizon joint-command tensors conditioned on… |
Tensor<T> System1Forward(Tensor<T> preprocessedObservation, Tensor<T> system2Latent) |
System 1 forward pass: runs the fast 80M visuomotor policy at every control tick (200 Hz target). |
Tensor<T> System2Forward(Tensor<T> preprocessedObservation, string instruction) |
System 2 forward pass: runs the full VLM (vision encoder + language decoder) and returns the semantic latent that conditions the fast System-1 controller. |