1. [Publications](/publications)
2. RGB-Only Reconstruction of Tabletop Scenes for Collision-Free Manipulator Control
 
 # RGB-Only Reconstruction of Tabletop Scenes for Collision-Free Manipulator Control 

  ![](/sites/default/files/styles/wide/public/publications/pic_0.png?itok=X2yjIlf2)

 We present a system for collision-free control of a robot manipulator that uses only RGB views of the world. Perceptual input of a tabletop scene is provided by multiple images of an RGB camera (without depth) that is either handheld or mounted on the robot end effector. A NeRF-like process is used to reconstruct the 3D geometry of the scene, from which the Euclidean full signed distance function (ESDF) is computed. A model predictive control algorithm is then used to control the manipulator to reach a desired pose while avoiding obstacles in the ESDF. We show results on a real dataset collected and annotated in our lab.



 ## Authors



Zhenggang Tang (Univ of Illinois)

[Balakumar Sundaralingam](/person/balakumar-sundaralingam)

[Jonathan Tremblay](/person/jonathan-tremblay)

[Bowen Wen](/person/bowen-wen)

[Ye Yuan](/person/ye-yuan)

[Stephen Tyree](/person/stephen-tyree)

[Charles Loop](/person/charles-loop)

Alexander Schwing (Univ of Illinois)

[Stan Birchfield](/person/stan-birchfield)

 

 

 ## Publication Date



Tuesday, May 16, 2023

 

 ## Published in



[ICRA 2023](https://www.icra2023.org/)

 

 ## Research Area



[Computer Vision](/research-area/computer-vision)

[Robotics](/research-area/robotics)

 

 

 ## External Links



[project page](https://ngp-mpc.github.io/)