Publications

Our publications provide insight into some of our leading-edge research.

Filters

Filters

Publication Year

Research Areas

Events

2025

Cosmos World Foundation Model Platform for Physical AI
Ming-Yu Liu, Many other contributors at https://d1qx31qr3h6wln.cloudfront.net/publications/NVIDIA%20Cosmos_4.pdf

2024

Diffusion-Reward Adversarial Imitation Learning
Chun-Mao Lai, Hsiang-Chun Wang, Ping-Chun Hsieh, Frank Wang, Min-Hung Chen, Shao-Hua Sun
MaskedMimic: Unified Physics-Based Character Control Through Masked Motion Inpainting
Chen Tessler, Kelly Guo, Ofir Nabati, Gal Chechik, Jason Peng
Differentiable GPU-Parallelized Task and Motion Planning
William Shen, Caelan Garrett, Ankit Goyal, Tucker Hermans, Fabio Ramos
SPIRE: Synergistic Planning, Imitation, and Reinforcement Learning for Long-Horizon Manipulation
Zihan Zhou, Animesh Garg, Dieter Fox, Caelan Garrett, Ajay Mandlekar
SkillGen: Automated Demonstration Generation for Efficient Skill Learning and Deployment
Caelan Garrett, Ajay Mandlekar, Bowen Wen, Dieter Fox
NOD-TAMP: Generalizable Long-Horizon Planning with Neural Object Descriptors
Shuo Cheng, Caelan Garrett, Ajay Mandlekar, Danfei Xu
HAMSTER: Hierarchical Action Models for Open-World Robot Manipulation
Yi Li, Yuquan Deng, Jesse Zhang, Joel Jang, Marius Memmel, Caelan Garrett, Fabio Ramos, Dieter Fox, Anqi Li, Abhishek Gupta, Ankit Goyal
Open-World Task and Motion Planning via Vision-Language Model Inferred Constraints
Nishanth Kumar, Fabio Ramos, Dieter Fox, Caelan Garrett
Guiding Long-Horizon Task and Motion Planning with Vision Language Models
Zhutian Yang, Caelan Garrett, Dieter Fox, Tomás Lozano-Pérez, Leslie Pack Kaelbling
Constructability-driven design of frame structures with state-space search methods
Yijiang Huang, Caelan Garrett, Caitlin Mueller
Interventional Data Generation for Robust and Data-Efficient Robot Imitation Learning
Ryan Hoque, Ajay Mandlekar, Caelan Garrett, Ken Goldberg, Dieter Fox
Proto-CLIP: Vision-Language Prototypical Network for Few-Shot Learning
Jishnu Jaykumar P, Kamalesh Palanisamy, Yu-Wei Chao, Xinya Du, Yu Xiang
DiMSam: Diffusion Models as Samplers for Task and Motion Planning under Partial Observability
Xiaolin Fang, Caelan Garrett, Clemens Eppner, Tomás Lozano-Pérez, Leslie Pack Kaelbling, Dieter Fox
Best Conference Paper Finalist
Best Student Paper Finalist
Learning to Move Like Professional Counter-Strike Players
David Durst, F. Xie, V. Sarukkai, Brennan Shacklett, Iuri Frosio, Chen Tessler, Joohwan Kim, C. Taylor, G. Bernstein, S. Choudhury, P. Hanrahan,, Kayvon Fatahalian
RVT-2: Learning Precise Manipulation from Few Examples
Ankit Goyal, Valts Blukis, Jie Xu, Yijie Guo, Yu-Wei Chao, Dieter Fox
Real-Time Anomaly Detection and Reactive Planning with Large Language Models
Rohan Sinha, Amine Elhafsi, Christopher Agia, Matthew Foutter, Edward Schmerling, Marco Pavone
FoundationPose: Unified 6D Pose Estimation and Tracking of Novel Objects
Bowen Wen, Wei Yang, Jan Kautz, Stan Birchfield
Neural Implicit Representation for Building Digital Twins of Unknown Articulated Objects
Yijia Weng, Bowen Wen, Jonathan Tremblay, Valts Blukis, Dieter Fox, Leo Guibas, Stan Birchfield
SynH2R: Synthesizing Hand-Object Motions for Learning Human-to-Robot Handovers
Sammy Christen, Lan Feng, Wei Yang, Yu-Wei Chao, Otmar Hilliges, Jie Song
SKT-Hang: Hanging Everyday Objects via Object-Agnostic Semantic Keypoint Trajectory Generation
Chia-Liang Kuo, Yu-Wei Chao, Yi-Ting Chen
One-Shot Transfer of Long-Horizon Extrinsic Manipulation Through Contact Retargeting
Albert Wu, Ruocheng Wang, Sirui Chen, Clemens Eppner, C Karen Liu

2023

RVT: Robotic View Transformer for 3D Object Manipulation
Ankit Goyal, Jie Xu, Yijie Guo, Valts Blukis, Yu-Wei Chao, Dieter Fox
Imitating Task and Motion Planning with Visuomotor Transformers
Murtaza Dalal, Ajay Mandlekar, Caelan Garrett, Ankur Handa, Ruslan Salakhutdinov, Dieter Fox
Human-In-The-Loop Task and Motion Planning for Imitation Learning
Ajay Mandlekar, Caelan Garrett, Danfei Xu, Dieter Fox
SCONE: A Food Scooping Robot Learning Framework with Active Perception
Yen-Ling Tai, Yu Chien Chiu, Yu-Wei Chao, Yi-Ting Chen
HANDAL: A Dataset of Real-World Manipulable Object Categories with Pose Annotations, Affordances, and Reconstructions
Andrew Guo, Bowen Wen, Jianhe Yuan, Jonathan Tremblay, Stephen Tyree, Jeff Smith, Stan Birchfield
VaPr: Variable-Precision Tensors to Accelerate Robot Motion Planning
Yu-Shun Hsiao, Siva Hari, Balakumar Sundaralingam, Jason Yik, Thierry Tambe, Charbel Sakr, Steve Keckler, Vijay Janapa Reddi
Bayesian Reparameterization of Reward-Conditioned Reinforcement Learning with Energy-based Models
Wenhao Ding, Gerry Che, Ding Zhao, Marco Pavone
Sequence-Based Plan Feasibility Prediction for Efficient Task and Motion Planning
Zhutian Yang, Caelan Garrett, Tomás Lozano-Pérez, Leslie Pack Kaelbling, Dieter Fox
Meta-Policy Learning over Plan Ensembles for Robust Articulated Object Manipulation
Constantinos Chamzas, Caelan Garrett, Balakumar Sundaralingam, Lydia Kavraki, Dieter Fox
StructDiffusion: Language-Guided Creation of Physically-Valid Structures using Unseen Objects
Weiyu Liu, Yilun Du, Tucker Hermans, Sonia Chernova, Chris Paxton