GR00TN1<T>
Models & Types · AiDotNet.VisionLanguage.Robotics
GR00T N1: NVIDIA’s open foundation model for generalist humanoid robots, combining a SigLIP + Eagle-2 vision-language System-2 reasoner with a flow-matching DiT action head as System 1 (NVIDIA 2025, arXiv:2503.14734).
Inheritance: NeuralNetworkBase<T> → VisionLanguageModelBase<T> → GR00TN1<T>
Implements: IVisionLanguageAction<T>
For Beginners
GR00T N1 is the first big-tech open humanoid brain. The fast policy doesn’t just predict joints directly — it predicts a noise-to-data flow, the same trick image generators like Stable Diffusion use. This lets it produce smooth, physically-plausible joint trajectories instead of jerky direct regressions.
How It Works
GR00T N1 is the first publicly released foundation model trained on the GR00T-1B humanoid dataset. Like Helix, it uses a dual-system architecture, but the System-1 policy is a flow-matching DiT (Lipman et al. 2023) rather than a direct regression head. At inference the model:
Paper-faithful pieces implemented here:
What is NOT verified in-session:
Example
var image = Tensor<double>.CreateRandom(1, 3, 32, 32);
var arch = new NeuralNetworkArchitecture<double>(
inputType: InputType.TwoDimensional,
taskType: NeuralNetworkTaskType.ImageClassification,
inputHeight: 224, inputWidth: 224, inputDepth: 3, outputSize: 52);
var gr00t = new GR00TN1<double>(arch, new GR00TN1Options());
// Single inference chunk (one S2 pass + Horizon flow-matching samples).
var jointCommands = gr00t.PredictAction(image, "set the table");
// Streaming control at the paper's 50 Hz S1 rate.
var runner = gr00t.CreateDualSystemRunner();
var currentInstruction = "stack the cups";
for (int tick = 0; tick < 200; tick++)
{
var currentImage = image; // your latest camera frame
var action = runner.Step(currentImage, currentInstruction);
// send `action` to the robot
}
Constructors
| Constructor | Summary |
|---|---|
GR00TN1(NeuralNetworkArchitecture<T> architecture, GR00TN1Options options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer) |
|
GR00TN1(NeuralNetworkArchitecture<T> architecture, string modelPath, GR00TN1Options options) |
Properties
| Property | Summary |
|---|---|
GR00TFlowMatchingActionHead<T> ActionHead { get; } |
The flow-matching action head used by PredictAction. |
int ActionDimension { get; } |
|
int DecoderEmbeddingDim { get; } |
|
int EmbeddingDimension { get; } |
|
int MaxGenerationLength { get; } |
|
string LanguageModelName { get; } |
Methods
| Method | Summary |
|---|---|
HelixDualSystemRunner<T> CreateDualSystemRunner() |
Wires this model into a HelixDualSystemRunner for streaming 50 Hz S1 control with periodic S2 re-invocation. |
ModelMetadata<T> GetModelMetadata() |
|
Tensor<T> EncodeImage(Tensor<T> image) |
|
Tensor<T> GenerateFromImage(Tensor<T> image, string prompt) |
|
Tensor<T> PredictAction(Tensor<T> observation, string instruction) |
Predicts a continuous joint-command horizon using the GR00T N1 dual-system protocol: one S2 pass (Eagle-2 VLM) + flow-matching Euler integration via ActionHead. |
Tensor<T> System1Velocity(Tensor<T> noisyAction, double t, Tensor<T> system2Latent) |
System-1 velocity field: takes a noisy action tensor at flow time t and the System-2 latent, runs the DiT-style velocity network, and returns the per-dim velocity. |
Tensor<T> System2Forward(Tensor<T> preprocessedObservation, string instruction) |
System-2 forward pass: SigLIP-style vision encoder + Eagle-2 LLM decoder + latent projection (paper §3.1). |