1. [Publications](/index.php/publications)
2. Deep Object Pose Estimation for Semantic Robotic Grasping of Household Objects
 
 # Deep Object Pose Estimation for Semantic Robotic Grasping of Household Objects

  ![](/sites/default/files/styles/wide/public/publications/forwebsite1_0.png?itok=kmCAzm93)

 Using synthetic data for training deep neural networks for robotic manipulation holds the promise of an almost unlimited amount of pre-labeled training data, generated safely out of harm's way. One of the key challenges of synthetic data, to date, has been to bridge the so-called *reality gap*, so that networks trained on synthetic data operate correctly when exposed to real-world data. We explore the reality gap in the context of 6-DoF pose estimation of known objects from a single RGB image. We show that for this problem the reality gap can be successfully spanned by a simple combination of domain randomized and photorealistic data. Using synthetic data generated in this manner, we introduce a one-shot deep neural network that is able to perform competitively against a state-of-the-art network trained on a combination of real and synthetic data. To our knowledge, this is the first deep network trained only on synthetic data that is able to achieve state-of-the-art performance on 6-DoF object pose estimation. Our network also generalizes better to novel environments including extreme lighting conditions, for which we show qualitative results. Using this network we demonstrate a real-time system estimating object poses with sufficient accuracy for real-world semantic grasping of known household objects in clutter by a real robot.



 ## Authors



[Jonathan Tremblay](/index.php/person/jonathan-tremblay)

Thang To (NVIDIA)

Bala Sundaralingam (NVIDIA, Univ. of Utah)

Yu Xiang (NVIDIA)

Dieter Fox (NVIDIA)

[Stan Birchfield](/index.php/person/stan-birchfield)

 

 

 ## Publication Date



Friday, September 28, 2018

 

 ## Published in



[Conference on Robot Learning (CoRL) 2018](http://www.robot-learning.org/)

 

 ## Research Area



[Computer Vision](/index.php/research-area/computer-vision)

[Robotics](/index.php/research-area/robotics)

 

 

 ## External Links



[arXiv paper](https://arxiv.org/abs/1809.10790)

[code](https://github.com/NVlabs/Deep_Object_Pose)

[video](https://youtu.be/yVGViBqWtBI)

[Jonathan's talk at CoRL](https://youtu.be/XteGx-iLHR4?t=10902)