Publications

Our publications provide insight into some of our leading-edge research.

Filters

Filters

Publication Year

Research Areas

Events

2025

ThinkAct: Vision-Language-Action Reasoning via Reinforced Visual Latent Planning
Chi-Pin Huang, Yueh-Hua Wu, Min-Hung Chen, Frank Wang, Fred Yang
Generalizable Domain Adaptation for Sim-and-Real Policy Co-Training
Shuo Cheng, Liqian Ma, Zhenyang Chen, Ajay Mandlekar, Caelan Garrett, Danfei Xu
ScheduleStream: Temporal Planning with Samplers for GPU-Accelerated Multi-Arm Task and Motion Planning & Scheduling
Caelan Garrett, Fabio Ramos
The Reality Gap in Robotics: Challenges, Solutions, and Best Practices
Elie Aljalbout, Jiaxu Xing, Angel Romero, Iretiayo Akinola, Caelan Garrett, Eric Heiden, Abhishek Gupta, Tucker Hermans, Yashraj Narang, Dieter Fox, Davide Scaramuzza, Fabio Ramos
ACGD: Visual Multitask Policy Learning with Asymmetric Critic Guided Distillation
Krishnan Srinivasan, Jie Xu, Henry Ang, Eric Heiden, Dieter Fox, Jeannette Bohg, Animesh Garg
Isaac Lab: A GPU Accelerated Simulation Framework For Multi-Modal Robot Learning
Mayank Mittal, Kelly Guo, Gavriel State, Spencer Huang
World Simulation With Video Foundation Models for Physical AI
Ming-Yu Liu,
VT-Refine: Learning Bimanual Assembly with Visuo-Tactile Feedback via Simulation Fine-Tuning
Binghao Huang, Jie Xu, Iretiayo Akinola, Wei Yang, Balakumar Sundaralingam, Rowland O'Flaherty, Dieter Fox, Xiaolong Wang, Arsalan Mousavian, Yu-Wei Chao, Yunzhu Li
Dexplore: Scalable Neural Control for Dexterous Manipulation from Reference-Scoped Exploration
Sirui Xu, Yu-Wei Chao, Liuyu Bian, Arsalan Mousavian, Yu-Xiong Wang, Liang-Yan Gui, Wei Yang
Neural Robot Dynamics
Jie Xu, Eric Heiden, Iretiayo Akinola, Dieter Fox, Miles Macklin, Yashraj Narang
Learning to Plan & Schedule with Reinforcement-Learned Bimanual Robot Skills
Weikang Wan, Fabio Ramos, Xuning Yang, Caelan Garrett
ReinforceGen: Hybrid Skill Policies with Automated Data Generation and Reinforcement Learning
Zihan Zhou, Animesh Garg, Ajay Mandlekar, Caelan Garrett
FoundationStereo: Zero-Shot Stereo Matching
Bowen Wen, Matthew Trepte, Joseph Aribido, Jan Kautz, Orazio Gallo, Stan Birchfield
Best Paper Nomination
RoboSpatial: Teaching Spatial Understanding to 2D and 3D Vision-Language Models for Robotics
Chan Hee Song, Valts Blukis, Jonathan Tremblay, Stephen Tyree, Yu Su, Stan Birchfield
Inference-Time Policy Steering through Human Interactions
Yanwei Wang, Lirui Wang, Yilun Du, Balakumar Sundaralingam, Xuning Yang, Yu-Wei Chao, Claudia Pérez D’Arpino , Dieter Fox, Julie Shah
TWIN: Two-handed Intelligent Benchmark for Bimanual Manipulation
Markus Grotz, Mohit Shridhar, Yu-Wei Chao, Tamim Asfour, Dieter Fox
SPOT: SE(3) Pose Trajectory Diffusion for Object-Centric Manipulation
Cheng-Chun Hsu, Bowen Wen, Jie Xu, Yashraj Narang, , Yuke Zhu, Joydeep Biswas, Stan Birchfield
Latent Action Pretraining from Videos
Seonghyeon Ye, Joel Jang, Byeongguk Jeon, Sejune Joo, Jianwei Yang, Baolin Peng, Ajay Mandlekar, Reuben Tan, Yu-Wei Chao, Yuchen Lin, Lars Liden, Kimin Lee, Jianfeng Gao, Luke Zettlemoyer, Dieter Fox, Minjoon Seo
Cosmos Transfer 1: World-to-World Transfer with Adaptive Multi-Control for Physical AI
Ming-Yu Liu
Cosmos-Reason 1: From Physical AI Common Sense to Embodied Decisions
Tsung-Yi Lin, Ming-Yu Liu
NVIDIA Isaac GR00T N1: An Open Foundation Model for Humanoid Robots
Yuke Zhu, Linxi "Jim" Fan, NVIDIA GEAR Team
Principles and guidelines for evaluating social robot navigation algorithms
Anthony Francis*, Claudia Pérez D’Arpino*, CHENGSHU LI, FEI XIA, ALEXANDRE ALAHI, RACHID ALAMI, ANIKET BERA, ABHIJAT BISWAS, JOYDEEP BISWAS, ROHAN CHANDRA, HAO-TIEN LEWIS CHIANG, MICHAEL EVERETT, SEHOON HA, JUSTIN HART, JONATHAN P. HOW, HARESH KARNAN, TSANG-WEI EDWARD LEE, LUIS J. MANSO, REUTH MIRSKY, SÖREN PIRK, PHANI TEJA SINGAMANENI, PETER STONE, ADA V. TAYLOR, PETER TRAUTMAN, NATHAN TSOI, MARYNEL VÁZQUEZ, XUESU XIAO, PENG XU, NAOKI YOKOYAMA, ALEXANDER TOSHEV, ROBERTO MARTÍN-MARTÍN
Cosmos World Foundation Model Platform for Physical AI
Ming-Yu Liu, Many other contributors at https://d1qx31qr3h6wln.cloudfront.net/publications/NVIDIA%20Cosmos_4.pdf
TacSL: A Library for Visuotactile Sensor Simulation and Learning
Iretiayo Akinola, Jie Xu, Jan Carius, Dieter Fox, Yashraj Narang

2024

FactorSim: Generative Simulation via Factorized Representation
Fan-Yun Sun, S. I. Harini, Angela Yi, Yihan Zhou, Alex Zook, Jonathan Tremblay, Logan Cross, Jiajun Wu, Nick Haber
Diffusion-Reward Adversarial Imitation Learning
Chun-Mao Lai, Hsiang-Chun Wang, Ping-Chun Hsieh, Frank Wang, Min-Hung Chen, Shao-Hua Sun
AdaDemo: Data-Efficient Demonstration Expansion for Generalist Robotic Agent
Tongzhou Mu, Yijie Guo, Jie Xu, Ankit Goyal, Hao Su, Dieter Fox, Animesh Garg
MaskedMimic: Unified Physics-Based Character Control Through Masked Motion Inpainting
Chen Tessler, Kelly Guo, Ofir Nabati, Gal Chechik, Jason Peng
Fast Explicit-Input Assistance for Teleoperation in Clutter
Nick Walker, Xuning Yang, Animesh Garg, Maya Cakmak, Dieter Fox, Claudia Pérez D’Arpino
Differentiable GPU-Parallelized Task and Motion Planning
William Shen, Caelan Garrett, Nishanth Kumar, Ankit Goyal, Tucker Hermans, Leslie Pack Kaelbling, Tomás Lozano-Pérez, Fabio Ramos
SPIRE: Synergistic Planning, Imitation, and Reinforcement Learning for Long-Horizon Manipulation
Zihan Zhou, Animesh Garg, Dieter Fox, Caelan Garrett, Ajay Mandlekar
SkillGen: Automated Demonstration Generation for Efficient Skill Learning and Deployment
Caelan Garrett, Ajay Mandlekar, Bowen Wen, Dieter Fox