Helix<T>

Models & Types · AiDotNet.VisionLanguage.Robotics

Helix: dual-system vision-language-action model for full-body humanoid control (Figure AI, 2025, arXiv:2502.07092).

Inheritance: NeuralNetworkBase<T> → VisionLanguageModelBase<T> → Helix<T>

Implements: IVisionLanguageAction<T>

For Beginners

Helix is the first robot brain that’s good enough to drive a real humanoid through dexterous tasks like loading a dishwasher. The trick is two networks: a smart-but- slow one decides intent (“pick up the red mug”) and a fast-but-narrow one figures out the exact motor commands needed to do it 200 times per second.

How It Works

Helix is the first generalist VLA model deployed on a real humanoid for whole upper-body dexterous manipulation. Its defining architectural choice is the dual-system, dual-rate split between a slow VLM reasoner and a fast visuomotor policy:

Paper-faithful pieces implemented here:

What is NOT verified in-session:

Example

var image = Tensor<double>.CreateRandom(1, 3, 32, 32);
var arch = new NeuralNetworkArchitecture<double>(
    inputType: InputType.TwoDimensional,
    taskType: NeuralNetworkTaskType.ImageClassification,
    inputHeight: 224, inputWidth: 224, inputDepth: 3, outputSize: 35);
var helix = new Helix<double>(arch, new HelixOptions());
            
// One-shot: predict a 16-tick horizon of joint commands.
var actions = helix.PredictAction(image, "stack the cups");
            
// Streaming: 200 Hz control loop with caller-owned timing.
var runner = helix.CreateDualSystemRunner();
var currentInstruction = "stack the cups";
for (int tick = 0; tick < 200; tick++)
{
    var currentImage = image;   // your latest camera frame
    var action = runner.Step(currentImage, currentInstruction);
    // send `action` to the robot
}

Constructors

Constructor Summary
Helix(NeuralNetworkArchitecture<T> architecture, HelixOptions options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer)
Helix(NeuralNetworkArchitecture<T> architecture, string modelPath, HelixOptions options)

Properties

Property Summary
int ActionDimension { get; }
int DecoderEmbeddingDim { get; }
int EmbeddingDimension { get; }
int MaxGenerationLength { get; }
int System1ToSystem2Ratio { get; } Default ratio of S1 to S2 invocations (paper §4.1: 200 Hz S1, 7–9 Hz S2 → ~22:1).
string LanguageModelName { get; }

Methods

Method Summary
HelixDualSystemRunner<T> CreateDualSystemRunner() Creates a HelixDualSystemRunner wired to this model’s S1/S2 callbacks for streaming-control use.
ModelMetadata<T> GetModelMetadata()
Tensor<T> EncodeImage(Tensor<T> image)
Tensor<T> GenerateFromImage(Tensor<T> image, string prompt)
Tensor<T> PredictAction(Tensor<T> observation, string instruction) Predicts ActionDimension × PredictionHorizon continuous joint commands using Helix’s dual-system inference path: one S2 invocation produces the semantic latent, then S1 rolls out PredictionHorizon joint-command tensors conditioned on…
Tensor<T> System1Forward(Tensor<T> preprocessedObservation, Tensor<T> system2Latent) System 1 forward pass: runs the fast 80M visuomotor policy at every control tick (200 Hz target).
Tensor<T> System2Forward(Tensor<T> preprocessedObservation, string instruction) System 2 forward pass: runs the full VLM (vision encoder + language decoder) and returns the semantic latent that conditions the fast System-1 controller.