GR00TN1<T>

Models & Types · AiDotNet.VisionLanguage.Robotics

GR00T N1: NVIDIA’s open foundation model for generalist humanoid robots, combining a SigLIP + Eagle-2 vision-language System-2 reasoner with a flow-matching DiT action head as System 1 (NVIDIA 2025, arXiv:2503.14734).

Inheritance: NeuralNetworkBase<T> → VisionLanguageModelBase<T> → GR00TN1<T>

Implements: IVisionLanguageAction<T>

For Beginners

GR00T N1 is the first big-tech open humanoid brain. The fast policy doesn’t just predict joints directly — it predicts a noise-to-data flow, the same trick image generators like Stable Diffusion use. This lets it produce smooth, physically-plausible joint trajectories instead of jerky direct regressions.

How It Works

GR00T N1 is the first publicly released foundation model trained on the GR00T-1B humanoid dataset. Like Helix, it uses a dual-system architecture, but the System-1 policy is a flow-matching DiT (Lipman et al. 2023) rather than a direct regression head. At inference the model:

Paper-faithful pieces implemented here:

What is NOT verified in-session:

Example

var image = Tensor<double>.CreateRandom(1, 3, 32, 32);
var arch = new NeuralNetworkArchitecture<double>(
    inputType: InputType.TwoDimensional,
    taskType: NeuralNetworkTaskType.ImageClassification,
    inputHeight: 224, inputWidth: 224, inputDepth: 3, outputSize: 52);
var gr00t = new GR00TN1<double>(arch, new GR00TN1Options());
            
// Single inference chunk (one S2 pass + Horizon flow-matching samples).
var jointCommands = gr00t.PredictAction(image, "set the table");
            
// Streaming control at the paper's 50 Hz S1 rate.
var runner = gr00t.CreateDualSystemRunner();
var currentInstruction = "stack the cups";
for (int tick = 0; tick < 200; tick++)
{
    var currentImage = image;   // your latest camera frame
    var action = runner.Step(currentImage, currentInstruction);
    // send `action` to the robot
}

Constructors

Constructor Summary
GR00TN1(NeuralNetworkArchitecture<T> architecture, GR00TN1Options options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer)
GR00TN1(NeuralNetworkArchitecture<T> architecture, string modelPath, GR00TN1Options options)

Properties

Property Summary
GR00TFlowMatchingActionHead<T> ActionHead { get; } The flow-matching action head used by PredictAction.
int ActionDimension { get; }
int DecoderEmbeddingDim { get; }
int EmbeddingDimension { get; }
int MaxGenerationLength { get; }
string LanguageModelName { get; }

Methods

Method Summary
HelixDualSystemRunner<T> CreateDualSystemRunner() Wires this model into a HelixDualSystemRunner for streaming 50 Hz S1 control with periodic S2 re-invocation.
ModelMetadata<T> GetModelMetadata()
Tensor<T> EncodeImage(Tensor<T> image)
Tensor<T> GenerateFromImage(Tensor<T> image, string prompt)
Tensor<T> PredictAction(Tensor<T> observation, string instruction) Predicts a continuous joint-command horizon using the GR00T N1 dual-system protocol: one S2 pass (Eagle-2 VLM) + flow-matching Euler integration via ActionHead.
Tensor<T> System1Velocity(Tensor<T> noisyAction, double t, Tensor<T> system2Latent) System-1 velocity field: takes a noisy action tensor at flow time t and the System-2 latent, runs the DiT-style velocity network, and returns the per-dim velocity.
Tensor<T> System2Forward(Tensor<T> preprocessedObservation, string instruction) System-2 forward pass: SigLIP-style vision encoder + Eagle-2 LLM decoder + latent projection (paper §3.1).