CoRL 2026

GRAIL

Generating Humanoid Loco-Manipulation
from World Models
Tianyi Xie1,2, Haotian Zhang1, Jinhyung Park1, Zi Wang1, Bowen Wen1, Jiefeng Li1, Xueting Li1, Qingwei Ben1, Haoyang Weng1, Yufei Ye1, David Minor1, Tingwu Wang1, Chenfanfu Jiang2, Sanja Fidler1, Jan Kautz1, Linxi Fan1, Yuke Zhu1, Zhengyi Luo1, Umar Iqbal1, Ye Yuan1
1NVIDIA     2UCLA
Motivation

Robotics Has a Data Problem

Teleoperation
Motion Capture
Operators teleoperating humanoid robots in a data-collection factory
Performers in marker suits in a motion-capture studio with wooden stairs and a ramp built for capture
Both need people to record every demonstration, so data grows only with the number of people recording it.
Teleoperation photo: The Standard (Hong Kong) · motion capture: Bones Studio SEED dataset
02
Motivation

Video World Models Generate Realistic Interactions

Lifting a moving box
Opening the fridge
Pushing an armchair
Pulling out a chair
Pushing a hand truck
Lifting a toolbox
From a text prompt alone: convincing human–object interactions, generated on demand.
Generated from text prompts with MiniMax-H3
What if these videos could become humanoid robot data?
A new paradigm
Generate robot data instead of collecting it
03
Key idea

Video World Models as Controllable Data Sources

Image input
Image input
Text prompt“… sits down on the armchair.”
Text prompt“… pushes the armchair to the wall.”
Text prompt“… walks to the window.”
✕Internet videos: no control over what we get
  • Camera cuts
  • Unknown, moving camera
  • Body out of frame
  • Occlusion
  • Meaningless motion
  • No interaction
✓Video world model: ask for what we need
Image-to-video with MiniMax-H3
04
Key idea

Video World Models as Controllable Data Sources

Image input
3D scene
render

Easy to edit

Assets · character · scene · camera

Known 3D information

Camera · metric scale · object geometry · body shape

05
Generated videos

Pick-Up with Data Augmentation

8× speed
06
Generated videos

The World Model Infers Object Affordance

07
Generated videos

Scene-Aware Interactions

08
Generated videos

Interacting with People

Toward human–robot interaction data
09
Reconstruct

Metric 4D Reconstruction

Human GEM-XObject FoundationPoseKnown scene
+ joint optimization of human, object and scene
10
Retarget

Meet Jason

Similar height and limb lengths to the G1: already robot-scaled
Unitree G11.32 m
Jason1.32 m
Retarget

Reconstruction → Unitree G1

11
Retarget

Multiple Embodiments

No spatial retargeting required
Unitree G1 · 1.32 m
Unitree H2 · 1.83 m
Booster T1 · 1.18 m
AgiBot X2 Ultra · 1.32 m
AgiBot A3 · 1.72 m
Each robot picks up the drill differently: we generate videos with the target embodiment in mind from the start.
12
Control

Grounding Kinematic Motions in Physics

RL motion trackingobject- & terrain-aware
Object- and terrain-aware RL motion tracking grounds every generated motion in physics.
13
Dataset

A Dataset Generated by GRAIL

~250K downloads on Hugging Face
huggingface.co/datasets/nvidia/PhysicalAI-Robotics-Locomanipulation-GRAIL
35,000+
physically validated loco-manipulation motions, and generating more
14
DEPLOY

Training Visual Policies in Simulation

Domain randomization · distilled from the trackers

Rendered egocentric RGB · head camera

Parallel simulated environments

15
Deploy · Real world

Real-World
Object Pick-Up

100% GRAIL training data
80%
success on unseen objectsZero-shot sim-to-real
16
Real world · indoor stairs
Real world · outdoor stairs
Simulation · GRAIL stair motions
Deploy

Sim-to-Real Stair Climbing

100%
success on lab stairsZero-shot sim-to-real
17
18
SUMMARY
GRAIL: a new paradigm for scaling robot learning
We envision a future where data is no longer the bottleneck.

“Walk up and down stairs.”

Text promptOne text prompt per skill
LLMcomposes the scenes
Video modelgenerates the behaviors
GRAILturns them into robot data and policies

“Pick up objects from the table or ground.”

Text promptOne text prompt per skill
LLMcomposes the scenes
Video modelgenerates the behaviors
GRAILturns them into robot data and policies
Thank you.
Our code and dataset are released. Feel free to check them out!
Code QR
Code
Dataset QR
Dataset