1. [Publications](/index.php/publications)
2. RVT: Robotic View Transformer for 3D Object Manipulation
 
 # RVT: Robotic View Transformer for 3D Object Manipulation

  ![](/sites/default/files/styles/wide/public/publications/Screenshot%202023-06-26%20064421.png?itok=2y2bU6ID)

 For 3D object manipulation, methods that build an explicit 3D representation perform better than those relying only on camera images. But using explicit 3D representations like voxels comes at a large computing cost, adversely affecting scalability. In this work, we propose RVT, a multi-view transformer for 3D manipulation that is both scalable and accurate. Some key features of RVT are an attention mechanism to aggregate information across views and re-rendering of the camera input from virtual views around the robot workspace. In simulations, we find that a single RVT model works well across 18 RLBench tasks with 249 task variations, achieving 26% higher relative success than the existing state-of-the-art method (PerAct). It also trains 36X faster than PerAct for achieving the same performance and achieves 2.3X the inference speed of PerAct. Further, RVT can perform a variety of manipulation tasks in the real world with just a few (∼10) demonstrations per task. Visual results, code, and trained model are provided at: https://robotic-view-transformer.github.io/.



 ## Authors



[Ankit Goyal](/index.php/person/ankit-goyal)

[Jie Xu](/index.php/person/jie-xu)

[Yijie Guo](/index.php/person/yijie-guo)

[Valts Blukis](/index.php/person/valts-blukis)

[Yu-Wei Chao](/index.php/person/yu-wei-chao)

Dieter Fox (NVIDIA)

 

 

 ## Publication Date



Monday, November 6, 2023

 

 ## Published in



[Conference on Robot Learning (CoRL) 2023](https://www.corl2023.org)

 

 ## Research Area



[Robotics](/index.php/research-area/robotics)

 

 

 ## External Links



[Project Page](https://robotic-view-transformer.github.io/)

 

 

 ## Uploaded Files



[RVT\_Arxiv\_2023.pdf](https://d1qx31qr3h6wln.cloudfront.net/publications/RVT_Arxiv_2023.pdf "Open file in new window")5.7 MB