Research Areas

Robotics

Associated Publications

2025

Inference-Time Policy Steering through Human Interactions
Yanwei Wang, Lirui Wang, Yilun Du, Balakumar Sundaralingam, Xuning Yang, Yu-Wei Chao, Claudia Pérez D’Arpino , Dieter Fox, Julie Shah
IEEE ICRA 2025
Cosmos-Reason 1: From Physical AI Common Sense to Embodied Decisions
Tsung-Yi Lin, Ming-Yu Liu
Cosmos Transfer 1: World-to-World Transfer with Adaptive Multi-Control for Physical AI
Ming-Yu Liu
Arxiv
NVIDIA Isaac GR00T N1: An Open Foundation Model for Humanoid Robots
Yuke Zhu, Linxi "Jim" Fan, NVIDIA GEAR Team
Principles and guidelines for evaluating social robot navigation algorithms
Anthony Francis*, Claudia Pérez D’Arpino*, CHENGSHU LI, FEI XIA, ALEXANDRE ALAHI, RACHID ALAMI, ANIKET BERA, ABHIJAT BISWAS, JOYDEEP BISWAS, ROHAN CHANDRA, HAO-TIEN LEWIS CHIANG, MICHAEL EVERETT, SEHOON HA, JUSTIN HART, JONATHAN P. HOW, HARESH KARNAN, TSANG-WEI EDWARD LEE, LUIS J. MANSO, REUTH MIRSKY, SÖREN PIRK, PHANI TEJA SINGAMANENI, PETER STONE, ADA V. TAYLOR, PETER TRAUTMAN, NATHAN TSOI, MARYNEL VÁZQUEZ, XUESU XIAO, PENG XU, NAOKI YOKOYAMA, ALEXANDER TOSHEV, ROBERTO MARTÍN-MARTÍN
ACM Transactions on Human-Robot Interaction
Cosmos World Foundation Model Platform for Physical AI
Ming-Yu Liu, Many other contributors at https://d1qx31qr3h6wln.cloudfront.net/publications/NVIDIA%20Cosmos_4.pdf

2024

FactorSim: Generative Simulation via Factorized Representation
Fan-Yun Sun, S. I. Harini, Angela Yi, Yihan Zhou, Alex Zook, Jonathan Tremblay, Logan Cross, Jiajun Wu, Nick Haber
NeurIPS 2024
Diffusion-Reward Adversarial Imitation Learning
Chun-Mao Lai, Hsiang-Chun Wang, Ping-Chun Hsieh, Frank Wang, Min-Hung Chen, Shao-Hua Sun
Neural Information Processing Systems (NeurIPS)
MaskedMimic: Unified Physics-Based Character Control Through Masked Motion Inpainting
Chen Tessler, Kelly Guo, Ofir Nabati, Gal Chechik, Jason Peng
SIGGRAPH Asia 2024
Fast Explicit-Input Assistance for Teleoperation in Clutter
Nick Walker, Xuning Yang, Animesh Garg, Maya Cakmak, Dieter Fox, Claudia Pérez D’Arpino
2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)
Differentiable GPU-Parallelized Task and Motion Planning
William Shen, Caelan Garrett, Ankit Goyal, Tucker Hermans, Fabio Ramos
CoRL 2024 Workshop on Differentiable Optimization
SkillGen: Automated Demonstration Generation for Efficient Skill Learning and Deployment
Caelan Garrett, Ajay Mandlekar, Bowen Wen, Dieter Fox
Conference on Robot Learning (CoRL)
NOD-TAMP: Generalizable Long-Horizon Planning with Neural Object Descriptors
Shuo Cheng, Caelan Garrett, Ajay Mandlekar, Danfei Xu
Conference on Robot Learning (CoRL)
SPIRE: Synergistic Planning, Imitation, and Reinforcement Learning for Long-Horizon Manipulation
Zihan Zhou, Animesh Garg, Dieter Fox, Caelan Garrett, Ajay Mandlekar
Conference on Robot Learning (CoRL)
Open-World Task and Motion Planning via Vision-Language Model Inferred Constraints
Nishanth Kumar, William Shen, Fabio Ramos, Dieter Fox, Tomás Lozano-Pérez, Leslie Pack Kaelbling, Caelan Garrett
CoRL 2024 Workshop on Language and Robot Learning Language as an Interface
HAMSTER: Hierarchical Action Models for Open-World Robot Manipulation
Yi Li, Yuquan Deng, Jesse Zhang, Joel Jang, Marius Memmel, Caelan Garrett, Fabio Ramos, Dieter Fox, Anqi Li, Abhishek Gupta, Ankit Goyal
CoRL 2024 Workshop on Language and Robot Learning Language as an Interface
Guiding Long-Horizon Task and Motion Planning with Vision Language Models
Zhutian Yang, Caelan Garrett, Dieter Fox, Tomás Lozano-Pérez, Leslie Pack Kaelbling
CoRL 2024 Workshop on Language and Robot Learning Language as an Interface
Constructability-driven design of frame structures with state-space search methods
Yijiang Huang, Caelan Garrett, Caitlin Mueller
Automation in Construction
DiMSam: Diffusion Models as Samplers for Task and Motion Planning under Partial Observability
Xiaolin Fang, Caelan Garrett, Clemens Eppner, Tomás Lozano-Pérez, Leslie Pack Kaelbling, Dieter Fox
IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)
Best Conference Paper Finalist, Best Student Paper Finalist
Interventional Data Generation for Robust and Data-Efficient Robot Imitation Learning
Ryan Hoque, Ajay Mandlekar, Caelan Garrett, Ken Goldberg, Dieter Fox
IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)
Proto-CLIP: Vision-Language Prototypical Network for Few-Shot Learning
Jishnu Jaykumar P, Kamalesh Palanisamy, Yu-Wei Chao, Xinya Du, Yu Xiang
IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) 2024
Learning to Move Like Professional Counter-Strike Players
David Durst, F. Xie, V. Sarukkai, Brennan Shacklett, Iuri Frosio, Chen Tessler, Joohwan Kim, C. Taylor, G. Bernstein, S. Choudhury, P. Hanrahan,, Kayvon Fatahalian
The 23rd ACM SIGGRAPH / Eurographics Symposium on Computer Animation (SCA 2024)
RVT-2: Learning Precise Manipulation from Few Examples
Ankit Goyal, Valts Blukis, Jie Xu, Yijie Guo, Yu-Wei Chao, Dieter Fox
Robotics: Science and Systems (RSS) 2024
Real-Time Anomaly Detection and Reactive Planning with Large Language Models
Rohan Sinha, Amine Elhafsi, Christopher Agia, Matthew Foutter, Edward Schmerling, Marco Pavone
RSS
Neural Implicit Representation for Building Digital Twins of Unknown Articulated Objects
Yijia Weng, Bowen Wen, Jonathan Tremblay, Valts Blukis, Dieter Fox, Leo Guibas, Stan Birchfield
CVPR 2024
FoundationPose: Unified 6D Pose Estimation and Tracking of Novel Objects
Bowen Wen, Wei Yang, Jan Kautz, Stan Birchfield
CVPR 2024
SKT-Hang: Hanging Everyday Objects via Object-Agnostic Semantic Keypoint Trajectory Generation
Chia-Liang Kuo, Yu-Wei Chao, Yi-Ting Chen
IEEE International Conference on Robotics and Automation (ICRA) 2024
SynH2R: Synthesizing Hand-Object Motions for Learning Human-to-Robot Handovers
Sammy Christen, Lan Feng, Wei Yang, Yu-Wei Chao, Otmar Hilliges, Jie Song
IEEE International Conference on Robotics and Automation (ICRA) 2024
One-Shot Transfer of Long-Horizon Extrinsic Manipulation Through Contact Retargeting
Albert Wu, Ruocheng Wang, Sirui Chen, Clemens Eppner, C Karen Liu
IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

2023

Human-In-The-Loop Task and Motion Planning for Imitation Learning
Ajay Mandlekar, Caelan Garrett, Danfei Xu, Dieter Fox
CoRL 2023
RVT: Robotic View Transformer for 3D Object Manipulation
Ankit Goyal, Jie Xu, Yijie Guo, Valts Blukis, Yu-Wei Chao, Dieter Fox
Conference on Robot Learning (CoRL) 2023
SCONE: A Food Scooping Robot Learning Framework with Active Perception
Yen-Ling Tai, Yu Chien Chiu, Yu-Wei Chao, Yi-Ting Chen
Conference on Robot Learning (CoRL) 2023
Imitating Task and Motion Planning with Visuomotor Transformers
Murtaza Dalal, Ajay Mandlekar, Caelan Garrett, Ankur Handa, Ruslan Salakhutdinov, Dieter Fox
CoRL 2023
VaPr: Variable-Precision Tensors to Accelerate Robot Motion Planning
Yu-Shun Hsiao, Siva Hari, Balakumar Sundaralingam, Jason Yik, Thierry Tambe, Charbel Sakr, Steve Keckler, Vijay Janapa Reddi
IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2023)
HANDAL: A Dataset of Real-World Manipulable Object Categories with Pose Annotations, Affordances, and Reconstructions
Andrew Guo, Bowen Wen, Jianhe Yuan, Jonathan Tremblay, Stephen Tyree, Jeff Smith, Stan Birchfield
IROS 2023
Bayesian Reparameterization of Reward-Conditioned Reinforcement Learning with Energy-based Models
Wenhao Ding, Gerry Che, Ding Zhao, Marco Pavone
International Conference on Machine Learning (ICML)
Meta-Policy Learning over Plan Ensembles for Robust Articulated Object Manipulation
Constantinos Chamzas, Caelan Garrett, Balakumar Sundaralingam, Lydia Kavraki, Dieter Fox
RSS 2023 Workshop on Learning for Task and Motion Planning
Sequence-Based Plan Feasibility Prediction for Efficient Task and Motion Planning
Zhutian Yang, Caelan Garrett, Tomás Lozano-Pérez, Leslie Pack Kaelbling, Dieter Fox
Robotics: Science and Systems (RSS)
StructDiffusion: Language-Guided Creation of Physically-Valid Structures using Unseen Objects
Weiyu Liu, Yilun Du, Tucker Hermans, Sonia Chernova, Chris Paxton
Robotics: Science and Systems (RSS) 2023
AnyTeleop: A General Vision-Based Dexterous Robot Arm-Hand Teleoperation System
Yuzhe Qin, Wei Yang, Binghao Huang, Karl Van Wyk, Hao Su, Xiaolong Wang, Yu-Wei Chao, Dieter Fox
Robotics: Science and Systems (RSS) 2023
Affordance Diffusion: Synthesizing Hand-Object Interactions
Yufei Ye, Xueting Li, Abhinav Gupta, Shalini De Mello, Stan Birchfield, Jiaming Song, Shubham Tulsiani, Sifei Liu
IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2023
Learning Human-to-Robot Handovers from Point Clouds
Sammy Christen, Wei Yang, Claudia Pérez D’Arpino , Otmar Hilliges, Dieter Fox, Yu-Wei Chao
IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 2023
Highlight
BundleSDF: Neural 6-DoF Tracking and 3D Reconstruction of Unknown Objects
Bowen Wen, Jonathan Tremblay, Valts Blukis, Stephen Tyree, Thomas Müller, Alex Evans, Dieter Fox, Jan Kautz, Stan Birchfield
CVPR 2023
Deep Learning Approaches to Grasp Synthesis: A Review
Rhys Newbury, Morris Gu, Lachlan Chumbley, Arsalan Mousavian, Clemens Eppner, Jürgen Leitner, Jeannette Bohg, Antonio Morales, Tamim Asfour, Danica Kragic, Dieter Fox, Akansel Cosgun
IEEE Transactions on Robotics
23 DoF Grasping Policies from a Raw Point Cloud
Martin Matak, Karl Van Wyk, Tucker Hermans
IEEE International Conference on Robotics and Automation (ICRA) Workshop on Geo…
Planning for Multi-Object Manipulation with Graph Neural Network Relational Classifiers
Yixuan Huang, Adam Conkey, Tucker Hermans
IEEE International Conference on Robotics and Automation (ICRA)
DeXtreme: Transfer of Agile In-Hand Manipulation from Simulation to Reality
Ankur Handa, Arthur Allshire, Viktor Makoviychuk, Aleksei Petrenko, Ritvik Singh, Jingzhou Liu, Denys Makoviichuk, Karl Van Wyk, Alexander Zhurkevich, Balakumar Sundaralingam, Yashraj Narang, Jean-Francois Lafleche, Dieter Fox, Gavriel State
https://arxiv.org/abs/2210.13702
FewSOL: A Dataset for Few-Shot Object Learning in Robotic Environments
Jishnu Jaykumar P, Yu-Wei Chao, Yu Xiang
IEEE International Conference on Robotics and Automation (ICRA) 2023
CuRobo: Parallelized Collision-Free Robot Motion Generation
Balakumar Sundaralingam, Siva Hari, Adam Fishman, Caelan Garrett, Karl Van Wyk, Valts Blukis, Alexander Millane, Helen Oleynikova, Ankur Handa, Fabio Ramos, Nathan Ratliff, Dieter Fox
IEEE International Conference on Robotics and Automation (ICRA)
Global and Reactive Motion Generation with Geometric Fabric Command Sequences
Weiming Zhi, Iretiayo Akinola, Karl Van Wyk, Nathan Ratliff, Fabio Ramos
IEEE International Conference on Robotics and Automation (ICRA) 2023
Parallel Inversion of Neural Radiance Fields for Robust Pose Estimation
Yunzhi Lin, Thomas Müller, Jonathan Tremblay, Bowen Wen, Stephen Tyree, Alex Evans, Patricio A. Vela, Stan Birchfield
ICRA 2023
ProgPrompt: Generating Situated Robot Task Plans Using Large Language Models
Ishika Singh, Valts Blukis, Arsalan Mousavian, Ankit Goyal, Danfei Xu, Jonathan Tremblay, Dieter Fox, Jesse Thomason, Animesh Garg
The International Conference on Robotics and Automation (ICRA)
DefGraspNets: Grasp Planning on 3D Fields with Graph Neural Nets
Isabella Huang, Yashraj Narang, Ruzena Bajcsy, Fabio Ramos, Tucker Hermans, Dieter Fox
IEEE International Conference on Robotics and Automation (ICRA)
IndustReal: Transferring Contact-Rich Assembly Tasks from Simulation to Reality
Bingjie Tang, Michael A. Lin, Iretiayo Akinola, Ankur Handa, Gaurav S. Sukhatme, Fabio Ramos, Dieter Fox, Yashraj Narang
Paper
RGB-Only Reconstruction of Tabletop Scenes for Collision-Free Manipulator Control
Zhenggang Tang, Balakumar Sundaralingam, Jonathan Tremblay, Bowen Wen, Ye Yuan, Stephen Tyree, Charles Loop, Alexander Schwing, Stan Birchfield
ICRA 2023
CabiNet: Scaling Neural Collision Detection for Object Rearrangement with Procedural Scene Generation
Adithya Murali, Arsalan Mousavian, Clemens Eppner, Adam Fishman, Dieter Fox
International Conference on Robotics and Automation (ICRA)

2022

Planning Visual-Tactile Precision Grasps via Complementary Use of Vision and Touch
Martin Matak, Tucker Hermans
IEEE Robotics and Automation Letters
Learning Robust Real-World Dexterous Grasping Policies via Implicit Shape Augmentation
Qiuyu Chen, Karl Van Wyk, Yu-Wei Chao, Wei Yang, Arsalan Mousavian, Abhishek Gupta, Dieter Fox
The Conference on Robot Learning (CoRL) 2022
MegaPose: 6D Pose Estimation of Novel Objects via Render & Compare
Yann Labbe, Lucas Manuelli, Arsalan Mousavian, Stephen Tyree, Stan Birchfield, Jonathan Tremblay, et al.
CoRL 2022
Motion Policy Networks
Adam Fishman, Adithya Murali, Clemens Eppner, Bryan Peele, Byron Boots, Dieter Fox
Conference on Robot Learning (CoRL), 2022
6-DoF Pose Estimation of Household Objects for Robotic Manipulation: An Accessible Dataset and Benchmark
Stephen Tyree, Jonathan Tremblay, Stan Birchfield, et al.
IROS 2022
Learning Perceptual Concepts by Bootstrapping from Human Queries
Andreea Bobu, Chris Paxton, Wei Yang, Balakumar Sundaralingam, Yu-Wei Chao, Maya Cakmak, Dieter Fox
RA-L / IROS 2022
Correcting Robot Plans with Natural Language Feedback
Pratyusha Sharma, Balakumar Sundaralingam, Valts Blukis, Chris Paxton, Tucker Hermans, Antonio Torralba, Jacob Andreas, Dieter Fox
Robotics Science and Systems (RSS) 2022
Ifor: Iterative flow minimization for robotic object rearrangement
Ankit Goyal, Arsalan Mousavian, Chris Paxton, Yu-Wei Chao, Brian Okorn, Jia Deng, Dieter Fox
IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 2022
HandoverSim: A Simulation Framework and Benchmark for Human-to-Robot Object Handovers
Yu-Wei Chao, Chris Paxton, Yu Xiang, Wei Yang, Balakumar Sundaralingam, Tao Chen, Adithya Murali, Maya Cakmak, Dieter Fox
IEEE International Conference on Robotics and Automation (ICRA) 2022
Factory: Fast Contact for Robotic Assembly
Yashraj Narang, Kier Storey, Iretiayo Akinola, Miles Macklin, Philipp Reist, Lukasz Wawrzyniak, Yunrong Guo, Adam Moravanszky, Gavriel State, Michelle Lu, Ankur Handa, Dieter Fox
Accelerated Policy Learning with Parallel Differentiable Simulation
Jie Xu, Viktor Makoviychuk, Yashraj Narang, Fabio Ramos, Wojciech Matusik, Animesh Garg, Miles Macklin
International Conference on Learning Representations (ICLR) 2022
Transform2Act: Learning a Transform-and-Control Policy for Efficient Agent Design
Ye Yuan, Yuda Song, Zhengyi Luo, Wen Sun, Kris Kitani
International Conference on Learning Representations (ICLR) 2022 (Oral)
Model Predictive Control for Fluid Human-to-Robot Handovers
Wei Yang, Balakumar Sundaralingam, Chris Paxton, Iretiayo Akinola, Yu-Wei Chao, Maya Cakmak, Dieter Fox
IEEE International Conference on Robotics and Automation (ICRA) 2022
DefGraspSim: Physics-Based Simulation of Grasp Outcomes on 3D Deformable Objects
Isabella Huang, Yashraj Narang, Clemens Eppner, Balakumar Sundaralingam, Miles Macklin, Ruzena Bajcsy, Tucker Hermans, Dieter Fox
IEEE Robotics and Automation Letters
PredictionNet: Real-Time Joint Probabilistic Traffic Prediction for Planning, Control, and Simulation
Alexey Kamenev, Lirui Wang, Ollin Boer Bohan, Ishwar Kulkarni, Bilal Kartal, Artem Molchanov, Stan Birchfield, David Nister, Nikolai Smolyanskiy
ICRA 2022
Cooperative Task and Motion Planning for Multi-Arm Assembly Systems
Jingkai Chen, Jiaoyang Li, Yijiang Huang, Caelan Garrett, Dawei Sun, Chuchu Fan, Andreas Hofmann, Caitlin Mueller, Sven Koenig, Brian Williams
https://arxiv.org/abs/2203.02475
Single-Stage Keypoint-Based Category-Level Object Pose Estimation from an RGB Image
Yunzhi Lin, Jonathan Tremblay, Stephen Tyree, Patricio A. Vela, Stan Birchfield
ICRA 2022
Keypoint-Based Category-Level Object Pose Tracking from an RGB Sequence with Uncertainty Estimation
Yunzhi Lin, Jonathan Tremblay, Stephen Tyree, Patricio A. Vela, Stan Birchfield
ICRA 2022

2021

Interpreting and predicting tactile signals for the SynTouch BioTac
Yashraj Narang, Balakumar Sundaralingam, Karl Van Wyk, Arsalan Mousavian, Dieter Fox
International Journal of Robotics Research (IJRR) 2021
Goal-Auxiliary Actor-Critic for 6D Robotic Grasping with Point Clouds
Lirui Wang, Yu Xiang, Wei Yang, Arsalan Mousavian, Dieter Fox
Conference on Robot Learning 2021
STORM: An Integrated Framework for Fast Joint-Space Model-Predictive Control for Reactive Manipulation
Mohak Bhardwaj, Balakumar Sundaralingam, Arsalan Mousavian, Nathan Ratliff, Dieter Fox, Fabio Ramos, Byron Boots
Conference on Robot Learning (CoRL) 2021
Selected for Oral Presentation
Joint Space Control via Deep Reinforcement Learning
Visak Kumar, David Hoeller, Balakumar Sundaralingam, Jonathan Tremblay, Stan Birchfield
IROS 2021
Multi-View Fusion for Multi-Level Robotic Scene Understanding
Yunzhi Lin, Jonathan Tremblay, Stephen Tyree, Patricio A. Vela, Stan Birchfield
IROS 2021
DexYCB: A Benchmark for Capturing Hand Grasping of Objects
Yu-Wei Chao, Wei Yang, Yu Xiang, Pavlo Molchanov, Ankur Handa, Jonathan Tremblay, Yashraj Narang, Karl Van Wyk, Umar Iqbal, Stan Birchfield, Jan Kautz, Dieter Fox
IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 2021
Sim-to-Real for Robotic Tactile Sensing via Physics-Based Simulation and Learned Latent Projections
Yashraj Narang, Balakumar Sundaralingam, Miles Macklin, Arsalan Mousavian, Dieter Fox
International Conference on Robotics and Automation (ICRA) 2021
ACRONYM: A Large-Scale Grasp Dataset Based on Simulation
Clemens Eppner, Arsalan Mousavian, Dieter Fox
2021 IEEE International Conference on Robotics and Automation (ICRA)
Alternative Paths Planner (APP) for Provably Fixed-time Manipulation Planning in Semi-structured Environments
Fahad Islam, Chris Paxton, Clemens Eppner, Bryan Peele, Maxim Likhachev, Dieter Fox
2021 IEEE International Conference on Robotics and Automation (ICRA)
Hierarchical Planning for Long-Horizon Manipulation with Geometric and Symbolic Scene Graphs
Yifeng Zhu, Jonathan Tremblay, Stan Birchfield, Yuke Zhu
ICRA 2021
Fast Uncertainty Quantification for Deep Object Pose Estimation
Guanya Shi, Yifeng Zhu, Jonathan Tremblay, Stan Birchfield, Fabio Ramos, Anima Anandkumar, Yuke Zhu
ICRA 2021
NViSII: A Scriptable Tool for Photorealistic Image Generation
Nathan Morrical, Jonathan Tremblay, Yunzhi Lin, Stephen Tyree, Stan Birchfield, Valerio Pascucci, Ingo Wald
SDG Workshop at ICLR 2021
RGB-D Local Implicit Function for Depth Completion of Transparent Objects
Luyang Zhu, Arsalan Mousavian, Yu Xiang, Hammad Mazhar, Jozef van Eenbergen, Shoubhik Debnath, Dieter Fox
CVPR 2021
Reactive Human-to-Robot Handovers of Arbitrary Objects
Wei Yang, Chris Paxton, Arsalan Mousavian, Yu-Wei Chao, Maya Cakmak, Dieter Fox
ICRA 2021
Best Paper in Human-Robot Interaction, ICRA 2021
Contact-GraspNet: Efficient 6-DoF Grasp Generation in Cluttered Scenes
Martin Sundermeyer, Arsalan Mousavian, Rudolph Triebel, Dieter Fox
ICRA 2021
Object Rearrangement Using Learned Implicit Collision Functions
Michael Danielczuk, Arsalan Mousavian, Clemens Eppner, Dieter Fox
ICRA 2021
Transferable Task Execution from Pixels through Deep Planning Domain Learning
Kei Kase, Chris Paxton, Hammad Mazhar, Tetsuya Ogata, Dieter Fox
ICRA 2020

2020

RMPflow: A Computational Graph for Automatic Motion Policy Generation
Ching-An Cheng, Mustafa Mukadam, Jan Issac, Stan Birchfield, Dieter Fox, Byron Boots, Nathan Ratliff
International Workshop on the Algorithmic Foundations of Robotics (WAFR) 2018
Contextual Reinforcement Learning of Visuo-tactile Multi-fingered Grasping Policies
Visak Kumar, Tucker Hermans, Dieter Fox, Stan Birchfield, Jonathan Tremblay
NeurIPS Workshop on Robot Learning
RMPflow: A Geometric Framework for Generation of Multi-Task Motion Policies
Ching-An Cheng, Mustafa Mukadam, Jan Issac, Stan Birchfield, Dieter Fox, Byron Boots, Nathan Ratliff
IEEE Transactions on Automation Science and Engineering (T-ASE)
Human Grasp Classification for Reactive Human-to-Robot Handovers
Wei Yang, Chris Paxton, Maya Cakmak, Dieter Fox
International Conference on Intelligent Robots and Systems (IROS), 2020
Collaborative Interaction Models for Optimized Human-Robot Teamwork
Adam Fishman, Chris Paxton, Wei Yang, Nathan Ratliff, Byron Boots, Dieter Fox
International Conference on Intelligent Robots and Systems (IROS) 2020
“Good Robot!”: Efficient Reinforcement Learning for Multi-Step Visual Tasks with Sim to Real Transfer
Andrew Hundt, Benjamin Killeen, Nicholas Greene, Hongtao Wu, Heeyeon Kwon, Chris Paxton, Gregory D Hager
IEEE Robotics and Automation Letters
Weakly-Supervised 3D Hand Pose Estimation via Biomechanical Constraints
Adrian Spurr, Umar Iqbal, Pavlo Molchanov, Otmar Hilliges, Jan Kautz
European Conference on Computer Vision, 2020
OCEAN: Online Task Inference for Compositional Tasks with Context Adaptation
Hongyu Ren, Yuke Zhu, Jure Leskovec, Anima Anandkumar, Animesh Garg
Conference on Uncertainty in Artificial Intelligence (UAI) 2020
Indirect Object-to-Robot Pose Estimation from an External Monocular RGB Camera
Jonathan Tremblay, Stephen Tyree, Terry Mosier, Stan Birchfield
IROS 2020
Automated Synthetic-to-Real Generalization
Wuyang Chen, Zhiding Yu, Zhangyang Wang, Anima Anandkumar
International Conference on Machine Learning (ICML) 2020
MVLidarNet: Real-Time Multi-Class Scene Understanding for Autonomous Driving Using Multiple Views
Ke Chen, Ryan Oldja, Nikolai Smolyanskiy, Stan Birchfield, Alexander Popov, David Wehr, Ibrahim Eden, Joachim Pehserl
IROS 2020
6-DOF Grasping for Target-driven Object Manipulation in Clutter
Adithyavairavan Murali, Arsalan Mousavian, Clemens Eppner, Chris Paxton, Dieter Fox
ICRA 2020
Best Paper Finalist in Robot Manipulation, ICRA 2020, Best Student Paper Finalist, ICRA 2020
Online replanning in belief space for partially observable task and motion problems
Caelan Garrett, Chris Paxton, Tomas Lozano-Perez, Leslie Kaebling, Dieter Fox
ICRA 2020
Self-supervised 6D Object Pose Estimation for Robot Manipulation
Xinke Deng, Yu Xiang, Arsalan Mousavian, Clemens Eppner, Timothy Bretl, Dieter Fox
2020 IEEE International Conference on Robotics and Automation (ICRA)
Motion reasoning for goal-based imitation learning
De-An Huang, Yu-Wei Chao, Chris Paxton, Xinke Deng, Li Fei-Fei, Juan Carlos Niebles, Animesh Garg, Dieter Fox
ICRA 2020
Camera-to-Robot Pose Estimation from a Single Image
Timothy E. Lee, Jonathan Tremblay, Thang To, Jia Cheng, Terry Mosier, Oliver Kroemer, Dieter Fox, Stan Birchfield
ICRA 2020
How to close sim-real gap? transfer with segmentation!
Mengyuan Yan, Qingyun Sun, Iuri Frosio, Stephen Tyree, Jan Kautz
arxiv
Toward Sim-to-Real Directional Semantic Grasping
Shariq Iqbal, Jonathan Tremblay, Thang To, Jia Cheng, Erik Leitch, Andy Campbell, Kirby Leung, Duncan McKay, Stan Birchfield
ICRA 2020
DexPilot: Vision Based Teleoperation of Dexterous Robotic Hand-Arm System
Ankur Handa, Karl Van Wyk, Wei Yang, Jacky Liang, Yu-Wei Chao, Qian Wan, Stan Birchfield, Nathan Ratliff, Dieter Fox
ICRA 2020

2019

6-DOF GraspNet: Variational Grasp Generation for Object Manipulation
Arsalan Mousavian, Clemens Eppner, Dieter Fox
ICCV 2019
A Billion Ways to Grasp: An Evaluation of Grasp Sampling Schemes on a Dense, Physics-based Grasp Data Set
Clemens Eppner, Arsalan Mousavian, Dieter Fox
The International Symposium of Robotics Research (ISSR)
Representing robot task plans as robust logical-dynamical systems
Chris Paxton, Clemens Eppner, Nathan Ratliff, Dieter Fox
IROS 2019
Putting Humans in a Scene: Learning Affordance in 3D Indoor Environments
Xueting Li, Sifei Liu, Kihwan Kim, Xiaolong Wang, Ming-Hsuan Yang, Jan Kautz
CVPR 2019
Robust Learning of Tactile Force Estimation through Robot Interaction
Balakumar Sundaralingam, Alexander Lambert, Ankur Handa, Byron Boots, Tucker Hermans, Stan Birchfield, Nathan Ratliff, Dieter Fox
ICRA 2019

2018

Fluidic Elastomer Actuators for Haptic Interactions in Virtual Reality
Jose Barreiros, Houston Claure, Bryan Peele, Omer Shapira, Josef Spjut, David Luebke, Malte Jung, Robert Shepherd
IEEE Robotics and Automation Letters
Deep Object Pose Estimation for Semantic Robotic Grasping of Household Objects
Jonathan Tremblay, Thang To, Bala Sundaralingam, Yu Xiang, Dieter Fox, Stan Birchfield
Conference on Robot Learning (CoRL) 2018
Learning Rigidity in Dynamic Scenes with a Moving Camera for 3D Motion Field Estimation
Zhaoyang Lv, Kihwan Kim, Alejandro Troccoli, Deqing Sun, James M. Rehg, Jan Kautz
European Conference on Computer Vision (ECCV 2018)
Simultaneous Edge Alignment and Learning
Zhiding Yu, Weiyang Liu, Yang Zou, Chen Feng, Srikumar Ramalingam, B. V. K. Vijaya Kumar, Jan Kautz
European Conference on Computer Vision (ECCV) 2018
HGMR: Hierarchical Gaussian Mixtures for Adaptive 3D Registration
Ben Eckart, Kihwan Kim, Jan Kautz
European Conference on Computer Vision (ECCV 2018)
Domain Adaptation for Semantic Segmentation via Class-Balanced Self-Training
Yang Zou, Zhiding Yu, B. V. K. Vijaya Kumar, Jinsong Wang
European Conference on Computer Vision (ECCV) 2018
EOE: Expected Overlap Estimation over Unstructured Point Cloud Data
Ben Eckart, Kihwan Kim, Jan Kautz
International Conference on 3D Vision (3DV) 2018
Region Growing Curriculum Generation for Reinforcement Learning
Artem Molchanov, Karol Hausman, Stan Birchfield, Gaurav Sukhatme
arXiv
Geometry-Aware Learning of Maps for Camera Localization
Samarth Brahmbhatt, Jinwei Gu, Kihwan Kim, James Hays, Jan Kautz
CVPR 2018 (Spotlight)
Falling Things: A Synthetic Dataset for 3D Object Detection and Pose Estimation
Jonathan Tremblay, Thang To, Stan Birchfield
CVPR 2018 Workshop on Real World Challenges and New Benchmarks for Deep Learnin…
PoseCNN: A Convolutional Neural Network for 6D Object Pose Estimation in Cluttered Scenes
Yu Xiang, Tanner Schmidt, Venkatraman Narayanan, Dieter Fox
Robotics: Science and Systems (RSS)
Synthetically Trained Neural Networks for Learning Human-Readable Plans from Real-World Demonstrations
Jonathan Tremblay, Thang To, Artem Molchanov, Stephen Tyree, Jan Kautz, Stan Birchfield
IEEE International Conference on Robotics and Automation (ICRA) 2018
A Variable Shape and Variable Stiffness Controller for Haptic Virtual Interactions
Benjamin C. Mac Murray, Bryan N. Peele, Patricia Xu, Josef Spjut, Omer Shapira, David Luebke, Robert F. Shepherd
RoboSoft
On the Importance of Stereo for Accurate Depth Estimation: An Efficient Semi-Supervised Deep Neural Network Approach
Nikolai Smolyanskiy, Alexey Kamenev, Stan Birchfield
CVPR 2018 Workshop on Autonomous Driving
Riemannian Motion Policies
Nathan Ratliff, Jan Issac, Daniel Kappler, Stan Birchfield, Dieter Fox
arXiv

2017

Sim-to-Real Transfer of Accurate Grasping with Eye-In-Hand Observations and Continuous Control
Mengyuan Yan, Iuri Frosio, Stephen Tyree, Jan Kautz
NIPS 2017 Workshop on Acting and Interacting in the Real World: Challenges in …
Toward Low-Flying Autonomous MAV Trail Navigation using Deep Neural Networks for Environmental Awareness
Nikolai Smolyanskiy, Alexey Kamenev, Jeffrey Smith, Stan Birchfield
IROS 2017

2016

Accelerated Generative Models for 3D Point Cloud Data
Ben Eckart, Kihwan Kim, Alejandro Troccoli, Alonzo Kelly, Jan Kautz
IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2016
Verification and Synthesis of Robust Control Barrier Functions: Multilevel Polynomial Optimization and Semidefinite Relaxation
Shucheng Kang, Yuxiao Chen, Heng Yang, Marco Pavone
IEEE Conference on Decision and Control (CDC)

Researchers

Adithya Murali
Ajay Mandlekar
Alán Aspuru-Guzik
Ankit Goyal
Anqi Li
Avnish Narayan
Balakumar Sundaralingam
Benjamin Eckart
Boris Ivanovic
Bowen Wen
Caelan Garrett
Chen-Hsuan Lin
Claudia Pérez D’Arpino
Danfei Xu
De-An Huang
Dennis Da
Ed Schmerling
Fabio Ramos
Fengyuan Hu
Fernando Castaneda
Guanzhi Wang
Heng Yang
Hugo Hadfield
Iuri Frosio
Yao Lu (Jason)
Jeff Smith
Jie Xu
Linxi "Jim" Fan
Jinwei Gu
Joel Jang
Jonathan Tremblay
Kaichun Mo
Karen Leung
Kris Wu
Lars Johannsmeier
Li Xu
Loic Magne
Marco Pavone
Nikita Cherniadev
Peter Karkus
Qinsheng Zhang
Rachel Luo
Rowland O'Flaherty
Scott Reed
Shuran Song
Siva Hari
Stan Birchfield
Stephen Tyree
Sushant Veer
Tucker Hermans
Valts Blukis
Wei Yang
Wenhao Ding
Wonmin Byeon
Xiaodong Yang
Xinshuo Weng
Xuan Li
Xuning Yang
Yashraj Narang
Ye Yuan
Yijie Guo
Yiming Li
Yinzhen Xu
Yongxin Chen
You Liang Tan
Yu Fang
Yu-Wei Chao
Yue Wang
Yuke Zhu
Yuqi Xie
Yuxiao Chen
Zhengyi Luo
Zhiding Yu