Introduction

We introduce Gr00t-N1.5, an upgraded version of the Gr00t-N1 foundation model for humanoid robots. With several architecture, data and modeling improvements, we find that N1.5 outperforms N1 on both simulated manipulation benchmarks and on the real GR-1 robot, detailed below. We expect users of N1.5 should observe better performance compared to N1, in particular improved generalization and better language following ability.

Model and Data Updates

Architecture

As with N1, Gr00t-N1.5 uses an NVIDIA Eagle VLM to encode text and visual observations. The vision-language embeddings from the VLM are then cross-attended to by the DiT which processes the state and noised actions.

The main differences from N1 are the following:

• The VLM model is frozen during both pretraining and finetuning.
• The adapter MLP connecting the vision encoder to the LLM is simplified and adds layer normalization to both visual and text token embeddings input to the LLM.

We found that these modifications greatly improved language following and generalization.

Improved VLM Grounding Capabilities

We have updated the VLM of Groot-N1.5, starting from Eagle 2.5 and tuned for better grounding capabilities and physical understanding. On RefCOCOg and our internal GEAR GR-1 grounding dataset with referring expressions, we observe that the N1.5 VLM model performs favorably compared to Qwen2.5-VL-3B - a comparable open-source model.

Left: Example annotations from our test dataset. Right: example model output.

Joint Policy Learning and World Modeling Objective

In addition to the flow matching loss used by N1, for N1.5 we add Future LAtent Representation Alignment (see FLARE paper). Rather than generatively modeling future frames, FLARE aligns the model with target future embeddings. We find that adding FLARE both improves policy performance and unlocks the ability to learn from human videos.

Training

We trained gr00t-N1.5 for 250K steps on 1K H100 GPUs with global batch size 16384. As in N1, we used AdamW with cosine learning rate schedule with warmup ratio 0.05. We used FLARE loss coefficient 0.2 for both pretraining and posttraining.

Our pretraining mixture included internal GR-1 data, OpenXE, simulated GR-1 (a.k.a. DexMG), neural trajectories from DreamGen, and AgiBot-Beta:

Distribution of training data in Gr00t-N1.5 pretraining.

Experimental Results

Architecture validation

In order to tune the model architecture for N1.5, we trained policies from scratch on two sim robot benchmarks requiring language following: Language Table and a set of five simulated GR-1 tasks requring language ("Sim GR-1 Language"). We find that the N1.5 architecture achieves significantly higher success rates on both benchmarks, indicating stronger language-conditioned control ability.

Select an object to see the performance comparison between GR00T N1 and N1.5:

Data-limited post-training in simulated environments

Following the Gr00t-N1 evaluation protocol, we evaluate N1.5's performance in data-limited post-training. In the case of Sim GR-1, we can evaluate both fewshot and 0-shot, since the the pretraining mixture includes other Sim GR-1 tasks with the same embodiment. We find that N1.5 is significantly better in the very low data regime (0-shot and 30 demos).

Real GR-1 language following

We add a simple language following task to the real GR-1 evaluation: two fruits are on a table and the robot is asked to place one of them onto a plate. The initial position of the target fruit is sampled to be closer to either the left or the right hand with 50% probability.

We find that N1.5 significantly improves over N1 in terms of its ability to follow language commands on the real GR-1 robot. Although both policies consistently pick and place some fruit onto the plate, N1.5 has a much higher language following rate, leading to a higher overall success rate.

Learning to manipulate novel objects from human ego videos

To evaluate the model's generalization ability, we evaluate pick and place performance using a set of 10 novel objects not seen during pretraining.

As shown in the FLARE paper, future latent representation alignment enables learning directly from human ego videos. This allows learning to manipulate novel objects from human videos and minimal robot demonstrations. Using N1.5, we found that this also works zero-shot.

Demonstrations using a novel object, captured from a GoPro (left) and the GR-1 robot (right).

Novel object generalization performance. We observe that N1.5 both performs better 0-shot, and also benefits from co-training with human videos.

Generalization to novel behaviors using Neural Trajectories

To generalize beyond the teleoperation data and enable humanoid robots to learn new tasks in new environments, we use DreamGen to generate synthetic robot data for training.

DreamGen 4-step pipeline.

By post-training GR00T-N1.5 on 50 DreamGen trajectories per behavior, we find that the model can learn to perform 8 new verbs:

We refer the reader to the DreamGen blog post for more details.

Discussion

Overall, we see that Groot-N1.5 is a significant improvement over Groot-N1. It achieves higher success rate, can use more diverse data sources, and has significantly improved language following capabilities. We attribute these improvements to the improved grounding capabilities, usage of the FLARE loss and the diverse data from DreamGen. The model will be open-sourced shortly, and we hope practitioners will observe improved results when finetuning Groot-N1.5 on their own robots.