Pick-up from table
The humanoid approaches, grasps, and lifts everyday household objects from tabletop surfaces.
CoRL 2026 Best Paper Award Finalist
Scaling humanoid loco-manipulation requires robot-compatible demonstrations across diverse objects, whole-body motions, and scene geometries, but teleoperation and motion capture are difficult to scale because each collection depends on physical setups, instrumented actors, and robot operation. We present GRAIL, a digital generation pipeline that remains fully virtual until deployment: it grounds video world models in 3D assets and simulator-ready scenes to synthesize interactions without rebuilding physical environments or teleoperating the robot.
Rather than reconstructing unconstrained in-the-wild videos, GRAIL starts from fully specified 3D configurations in which object geometry, camera parameters, metric scale, environment depth, and a robot-proportioned character are known before video generation and reused during reconstruction. This privileged setup better conditions 4D recovery, allowing model-based object tracking, human motion estimation, and interaction-aware optimization to reconstruct metric 4D human-object interaction (HOI) trajectories with reduced depth ambiguity and morphology mismatch. We retarget the recovered motions to a humanoid robot and train complementary task-general trackers: an object-aware latent adaptor for manipulation and a scene-aware tracker for terrain traversal.
GRAIL now includes over 35,000 physically validated motions, covering both object-centric interactions and scene-centric whole-body motion. The gallery below shows object manipulation and scene-aware locomotion as two complementary capability blocks.
Object-centric motions couple navigation, grasp geometry, and contact-rich whole-body control.
The humanoid approaches, grasps, and lifts everyday household objects from tabletop surfaces.
Ground-level pick-up requires squatting, reaching, grasping, and returning to a stable standing posture.
Larger and constrained objects require coordinated approach, grasp, carry, push, and repositioning motions.
Scene-centric motions stress body alignment, foot placement, and terrain-conditioned balance.
Chair interactions require approach, body alignment, and settling into a seated posture.
Curbs exercise adaptive stepping and balance recovery over raised platforms.
Slope traversal tests balance and gait adaptation over inclined terrain.
Staircases require precise foot placement, sustained balance, and scene-conditioned whole-body control.
Generated motions adapt to diverse stair geometries, materials, and step heights.
We deploy policies trained with GRAIL-generated data on a real Unitree G1, covering egocentric RGB-based pick-up and stair-climbing.
GRAIL renders egocentric RGB videos from generated grasping demonstrations, providing visual-policy training data before transferring the policy to real-world deployment.
We train egocentric RGB policies on GRAIL-generated data and deploy them on a Unitree G1 for real-world object pick-up and stair-climbing.
GRAIL grounds video world models in fully specified 3D scenes. It first renders a scene with known object geometry, camera parameters, metric scale, and a robot-proportioned character, then uses a video world model to roll out a human-object interaction. It recovers metric 4D motion within this known scene and retargets it to the Unitree G1, where complementary task-general trackers turn the references into robot-action data for downstream visual policy learning.
Recovered human and object motion stays aligned with the generated interaction video within the known metric scene, across manipulation, stair traversal, and sitting.
A continuous view of generated humanoid behaviors across object manipulation, sitting, and terrain traversal, from individual interactions to a broad collection of motions.
If you find GRAIL useful in your research, please cite:
@inproceedings{grail2026,
title = {{GRAIL}: Generating Humanoid Loco-Manipulation from World Models},
author = {Tianyi Xie and Haotian Zhang and Jinhyung Park and Zi Wang and
Bowen Wen and Jiefeng Li and Xueting Li and Qingwei Ben and
Haoyang Weng and Yufei Ye and David Minor and Tingwu Wang and
Chenfanfu Jiang and Sanja Fidler and Jan Kautz and Linxi Fan and
Yuke Zhu and Zhengyi Luo and Umar Iqbal and Ye Yuan},
booktitle = {Conference on Robot Learning (CoRL)},
year = {2026},
url = {https://arxiv.org/abs/2606.05160},
}