Jonathan Tremblay ![](/sites/default/files/person/profile.jpg) Jonathan is a research scientist at NVIDIA. His research interests are in computer vision, synthetic data, and reinforcement learning for robotics applications. At NVIDIA, Jonathan has focused on using synthetic data to train object detector, object pose estimation, few shot learning, etc. Jonathan's goal is to create robust and accessible computer vision systems for robotists to use on there system. Prior to joining NVIDIA, Jonathan received Ph.D. in computer science from McGill University. Research Area(s) [Robotics](/index.php/research-area/robotics) Main Field of Interest [Computer Vision](/index.php/research-area/computer-vision) Google Scholar [https://scholar.google.ca/citations?user=zeS5UJEAAAAJ&hl=en](https://scholar.google.ca/citations?user=zeS5UJEAAAAJ&hl=en) ### Publications ### 2026 [3D-GENERALIST: Vision-Language-Action Models for Crafting 3D Worlds](/publication/2026-03_3d-generalist-vision-language-action-models-crafting-3d-worlds) Fan-Yun Sun, Shengguang Wu, Christian Jacobsen, Thomas Yim, Haoming Zou, [Alex Zook](/person/alex-zook), Shangru Li, Yu-Hsin Chou, Ethem Can, Xunlei Wu, Clemens Eppner, [Valts Blukis](/person/valts-blukis), [Jonathan Tremblay](/person/jonathan-tremblay), Jiajun Wu, [Stan Birchfield](/person/stan-birchfield), Nick Haber [International Conference on 3D Vision 2026](https://3dvconf.github.io/2026/) ### 2025 [RoboArena: Distributed Real-World Evaluation of Generalist Robot Policies](/publication/2025-09_roboarena-distributed-real-world-evaluation-generalist-robot-policies) Pranav Atreya, Karl Pertsch, Tony Lee, Moo Jin Kim, Arhan Jain, Artur Kuramshin, Clemens Eppner, Cyrus Neary, Edward Hu, [Fabio Ramos](/person/fabio-ramos), [Jonathan Tremblay](/person/jonathan-tremblay), Kanav Arora, Kirsty Ellis, Luca Macesanu, Marcel Torne Villasevil, Matthew Leonard, Meedeum Cho, Ozgur Aslan, Shivin Dass, Jie Wang, William Reger, Xingfang Yuan, [Xuning Yang](/person/xuning-yang), Abhishek Gupta, Dinesh Jayaraman, Glen Berseth, Kostas Daniilidis, Roberto Martin-Martin, Youngwoon Lee, Percy Liang, Chelsea Finn, Sergey Levine [CoRL 2025](https://arxiv.org/abs/2506.18123) [Fly, Fail, Fix: Iterative Game Repair with Reinforcement Learning and Large Multimodal Models](/publication/2025-08_fly-fail-fix-iterative-game-repair-reinforcement-learning-and-large-multimodal) [Alex Zook](/person/alex-zook), [Josef Spjut](/person/josef-spjut), [Jonathan Tremblay](/person/jonathan-tremblay) [Reinforcement Learning and Video Games Workshop @ RLC 2025](https://sites.google.com/view/rlvg-workshop-2025/home) [Towards a VLM Benchmark for Simulated Robotics](/publication/2025-06_towards-vlm-benchmark-simulated-robotics) [Xuning Yang](/person/xuning-yang), Clemens Eppner, [Valts Blukis](/person/valts-blukis), [Peter Belcak](/person/peter-belcak), [Stephen Tyree](/person/stephen-tyree), [Stan Birchfield](/person/stan-birchfield), [Fabio Ramos](/person/fabio-ramos), [Jonathan Tremblay](/person/jonathan-tremblay) [RSS 2025 Workshop on Large Foundation Models for Interactive Robot Learning](https://lfmrss2025.weebly.com/) [Robot policy evaluation for sim-to-real transfer: A benchmarking perspective](/publication/2025-06_robot-policy-evaluation-sim-real-transfer-benchmarking-perspective) [Xuning Yang](/person/xuning-yang), Clemens Eppner, [Jonathan Tremblay](/person/jonathan-tremblay), Dieter Fox, [Stan Birchfield](/person/stan-birchfield), [Fabio Ramos](/person/fabio-ramos) [RSS 2025 Workshop on Robot Evaluations](https://sites.google.com/stanford.edu/robot-evaluation-rss-2025/home) [GRS: Generating robotic simulation tasks from real-world images](/publication/2025-06_grs-generating-robotic-simulation-tasks-real-world-images-0) [Alex Zook](/person/alex-zook), [Josef Spjut](/person/josef-spjut), [Jonathan Tremblay](/person/jonathan-tremblay) [CVPR 2025](https://cvpr.thecvf.com/Conferences/2025) [RoboSpatial: Teaching Spatial Understanding to 2D and 3D Vision-Language Models for Robotics](/publication/2025-06_robospatial-teaching-spatial-understanding-2d-and-3d-vision-language-models) Chan Hee Song, [Valts Blukis](/person/valts-blukis), [Jonathan Tremblay](/person/jonathan-tremblay), [Stephen Tyree](/person/stephen-tyree), Yu Su, [Stan Birchfield](/person/stan-birchfield) [CVPR 2025](https://cvpr.thecvf.com/) ### 2024 [FactorSim: Generative Simulation via Factorized Representation](/publication/2024-12_factorsim-generative-simulation-factorized-representation) Fan-Yun Sun, S. I. Harini, Angela Yi, Yihan Zhou, [Alex Zook](/person/alex-zook), [Jonathan Tremblay](/person/jonathan-tremblay), Logan Cross, Jiajun Wu, Nick Haber [NeurIPS 2024](https://neurips.cc/Conferences/2024) [Neural Implicit Representation for Building Digital Twins of Unknown Articulated Objects](/publication/2024-06_neural-implicit-representation-building-digital-twins-unknown-articulated) Yijia Weng, [Bowen Wen](/person/bowen-wen), [Jonathan Tremblay](/person/jonathan-tremblay), [Valts Blukis](/person/valts-blukis), Dieter Fox, Leo Guibas, [Stan Birchfield](/person/stan-birchfield) [CVPR 2024](https://cvpr.thecvf.com/Conferences/2024) [NeRFDeformer: NeRF Transformation from a Single View via 3D Scene Flows](/index.php/publication/2024-06_nerfdeformer-nerf-transformation-single-view-3d-scene-flows) Zhenggang Tang, Zhongzheng Ren, Xiaoming Zhao, [Bowen Wen](/index.php/person/bowen-wen), [Jonathan Tremblay](/index.php/person/jonathan-tremblay), [Stan Birchfield](/index.php/person/stan-birchfield), Alexander Schwing [CVPR 2024](https://cvpr.thecvf.com/Conferences/2024) ### 2023 [HANDAL: A Dataset of Real-World Manipulable Object Categories with Pose Annotations, Affordances, and Reconstructions](/publication/2023-10_handal-dataset-real-world-manipulable-object-categories-pose-annotations) Andrew Guo, [Bowen Wen](/person/bowen-wen), Jianhe Yuan, [Jonathan Tremblay](/person/jonathan-tremblay), [Stephen Tyree](/person/stephen-tyree), [Jeff Smith](/person/jeff-smith), [Stan Birchfield](/person/stan-birchfield) [IROS 2023](https://ieee-iros.org/) [TTA-COPE: Test-Time Adaptation for Category-Level Object Pose Estimation](/publication/2023-06_tta-cope-test-time-adaptation-category-level-object-pose-estimation) Taeyeop Lee, [Jonathan Tremblay](/person/jonathan-tremblay), [Valts Blukis](/person/valts-blukis), [Bowen Wen](/person/bowen-wen), Byeong-Uk Lee, Inkyu Shin, [Stan Birchfield](/person/stan-birchfield), In So Kweon, Kuk-Jin Yoon [CVPR 2023](https://cvpr2023.thecvf.com/) [BundleSDF: Neural 6-DoF Tracking and 3D Reconstruction of Unknown Objects](/publication/2023-06_bundlesdf-neural-6-dof-tracking-and-3d-reconstruction-unknown-objects) [Bowen Wen](/person/bowen-wen), [Jonathan Tremblay](/person/jonathan-tremblay), [Valts Blukis](/person/valts-blukis), [Stephen Tyree](/person/stephen-tyree), [Thomas Müller](/person/thomas-muller), Alex Evans, Dieter Fox, [Jan Kautz](/person/jan-kautz), [Stan Birchfield](/person/stan-birchfield) [CVPR 2023](https://cvpr2023.thecvf.com/) [Parallel Inversion of Neural Radiance Fields for Robust Pose Estimation](/publication/2023-05_parallel-inversion-neural-radiance-fields-robust-pose-estimation) Yunzhi Lin, [Thomas Müller](/person/thomas-muller), [Jonathan Tremblay](/person/jonathan-tremblay), [Bowen Wen](/person/bowen-wen), [Stephen Tyree](/person/stephen-tyree), Alex Evans, Patricio A. Vela, [Stan Birchfield](/person/stan-birchfield) [ICRA 2023](https://www.icra2023.org/) [ProgPrompt: Generating Situated Robot Task Plans Using Large Language Models](/publication/2023-05_progprompt-generating-situated-robot-task-plans-using-large-language-models) Ishika Singh, [Valts Blukis](/person/valts-blukis), Arsalan Mousavian, [Ankit Goyal](/person/ankit-goyal), [Danfei Xu](/person/danfei-xu), [Jonathan Tremblay](/person/jonathan-tremblay), Dieter Fox, Jesse Thomason, Animesh Garg [The International Conference on Robotics and Automation (ICRA)](https://www.icra2023.org/welcome) [RGB-Only Reconstruction of Tabletop Scenes for Collision-Free Manipulator Control ](/publication/2023-05_rgb-only-reconstruction-tabletop-scenes-collision-free-manipulator-control) Zhenggang Tang, [Balakumar Sundaralingam](/person/balakumar-sundaralingam), [Jonathan Tremblay](/person/jonathan-tremblay), [Bowen Wen](/person/bowen-wen), [Ye Yuan](/person/ye-yuan), [Stephen Tyree](/person/stephen-tyree), [Charles Loop](/person/charles-loop), Alexander Schwing, [Stan Birchfield](/person/stan-birchfield) [ICRA 2023](https://www.icra2023.org/) ### 2022 [MegaPose: 6D Pose Estimation of Novel Objects via Render & Compare](/publication/2022-12_megapose-6d-pose-estimation-novel-objects-render-compare) Yann Labbe, Lucas Manuelli, Arsalan Mousavian, [Stephen Tyree](/person/stephen-tyree), [Stan Birchfield](/person/stan-birchfield), [Jonathan Tremblay](/person/jonathan-tremblay), et al. [CoRL 2022](https://corl2022.org/) [6-DoF Pose Estimation of Household Objects for Robotic Manipulation: An Accessible Dataset and Benchmark](/publication/2022-11_6-dof-pose-estimation-household-objects-robotic-manipulation-accessible-dataset) [Stephen Tyree](/person/stephen-tyree), [Jonathan Tremblay](/person/jonathan-tremblay), [Stan Birchfield](/person/stan-birchfield), et al. [IROS 2022](https://iros2022.org/) [Variable Bitrate Neural Fields](/vbnf) Towaki Takikawa, Alex Evans, [Jonathan Tremblay](/person/jonathan-tremblay), [Thomas Müller](/person/thomas-muller), Morgan McGuire, Alec Jacobson, Sanja Fidler [ACM SIGGRAPH 2022 Conference Proceedings](https://s2022.siggraph.org/) [Efficient Geometry-aware 3D Generative Adversarial Networks](/publication/2022-06_efficient-geometry-aware-3d-generative-adversarial-networks) Eric R. Chan, Connor Z. Lin, Matthew A. Chan, [Koki Nagano](/person/koki-nagano), Boxiao Pan, [Shalini De Mello](/person/shalini-de-mello), Orazio Gallo, Leonidas Guibas, [Jonathan Tremblay](/person/jonathan-tremblay), Sameh Khamis, [Tero Karras](/person/tero-karras), Gordon Wetzstein [IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2022](https://cvpr2022.thecvf.com/) Oral [Single-Stage Keypoint-Based Category-Level Object Pose Estimation from an RGB Image](/publication/2022-02_single-stage-keypoint-based-category-level-object-pose-estimation-rgb-image) Yunzhi Lin, [Jonathan Tremblay](/person/jonathan-tremblay), [Stephen Tyree](/person/stephen-tyree), Patricio A. Vela, [Stan Birchfield](/person/stan-birchfield) ICRA 2022 [Keypoint-Based Category-Level Object Pose Tracking from an RGB Sequence with Uncertainty Estimation](/publication/2022-01_keypoint-based-category-level-object-pose-tracking-rgb-sequence-uncertainty) Yunzhi Lin, [Jonathan Tremblay](/person/jonathan-tremblay), [Stephen Tyree](/person/stephen-tyree), Patricio A. Vela, [Stan Birchfield](/person/stan-birchfield) ICRA 2022 [RTMV: A Ray-Traced Multi-View Synthetic Dataset for Novel View Synthesis ](/publication/2022-01_rtmv-ray-traced-multi-view-synthetic-dataset-novel-view-synthesis) [Jonathan Tremblay](/person/jonathan-tremblay), Moustafa Meshry, [Stan Birchfield](/person/stan-birchfield), Alex Evans, [Jan Kautz](/person/jan-kautz), [Alex Keller](/person/alex-keller), Sameh Khamis, [Charles Loop](/person/charles-loop), Nate Morrical, [Thomas Müller](/person/thomas-muller), [Koki Nagano](/person/koki-nagano), Towaki Takikawa, [Stan Birchfield](/person/stan-birchfield) [ECCV 2022 Workshop on Learning to Generate 3D Shapes and Scenes](https://learn3dg.github.io/) ### 2021 [Joint Space Control via Deep Reinforcement Learning](/publication/2021-09_joint-space-control-deep-reinforcement-learning) Visak Kumar, David Hoeller, [Balakumar Sundaralingam](/person/balakumar-sundaralingam), [Jonathan Tremblay](/person/jonathan-tremblay), [Stan Birchfield](/person/stan-birchfield) [IROS 2021](https://www.iros2021.org/) [Multi-View Fusion for Multi-Level Robotic Scene Understanding](/publication/2021-09_multi-view-fusion-multi-level-robotic-scene-understanding) Yunzhi Lin, [Jonathan Tremblay](/person/jonathan-tremblay), [Stephen Tyree](/person/stephen-tyree), Patricio A. Vela, [Stan Birchfield](/person/stan-birchfield) [IROS 2021](https://www.iros2021.org/) [DexYCB: A Benchmark for Capturing Hand Grasping of Objects](/publication/2021-06_dexycb-benchmark-capturing-hand-grasping-objects) [Yu-Wei Chao](/person/yu-wei-chao), [Wei Yang](/person/wei-yang), Yu Xiang, [Pavlo Molchanov](/person/pavlo-molchanov), Ankur Handa, [Jonathan Tremblay](/person/jonathan-tremblay), [Yashraj Narang](/person/yashraj-narang), Karl Van Wyk, [Umar Iqbal](/person/umar-iqbal), [Stan Birchfield](/person/stan-birchfield), [Jan Kautz](/person/jan-kautz), Dieter Fox [IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 2021](http://cvpr2021.thecvf.com) [Hierarchical Planning for Long-Horizon Manipulation with Geometric and Symbolic Scene Graphs](/publication/2021-05_hierarchical-planning-long-horizon-manipulation-geometric-and-symbolic-scene) Yifeng Zhu, [Jonathan Tremblay](/person/jonathan-tremblay), [Stan Birchfield](/person/stan-birchfield), [Yuke Zhu](/person/yuke-zhu) [ICRA 2021](https://www.ieee-icra.org/) [Fast Uncertainty Quantification for Deep Object Pose Estimation](/publication/2021-05_fast-uncertainty-quantification-deep-object-pose-estimation) Guanya Shi, Yifeng Zhu, [Jonathan Tremblay](/person/jonathan-tremblay), [Stan Birchfield](/person/stan-birchfield), [Fabio Ramos](/person/fabio-ramos), Anima Anandkumar, [Yuke Zhu](/person/yuke-zhu) [ICRA 2021](https://www.ieee-icra.org/) [NViSII: A Scriptable Tool for Photorealistic Image Generation](/publication/2021-05_nvisii-scriptable-tool-photorealistic-image-generation) Nathan Morrical, [Jonathan Tremblay](/person/jonathan-tremblay), Yunzhi Lin, [Stephen Tyree](/person/stephen-tyree), [Stan Birchfield](/person/stan-birchfield), Valerio Pascucci, Ingo Wald SDG Workshop at ICLR 2021 ### 2020 [Contextual Reinforcement Learning of Visuo-tactile Multi-fingered Grasping Policies](/publication/2020-11_contextual-reinforcement-learning-visuo-tactile-multi-fingered-grasping) Visak Kumar, [Tucker Hermans](/person/tucker-hermans), Dieter Fox, [Stan Birchfield](/person/stan-birchfield), [Jonathan Tremblay](/person/jonathan-tremblay) [NeurIPS Workshop on Robot Learning](http://www.robot-learning.ml/2020/) [Indirect Object-to-Robot Pose Estimation from an External Monocular RGB Camera](/publication/2020-07_indirect-object-robot-pose-estimation-external-monocular-rgb-camera) [Jonathan Tremblay](/person/jonathan-tremblay), [Stephen Tyree](/person/stephen-tyree), Terry Mosier, [Stan Birchfield](/person/stan-birchfield) IROS 2020 [Camera-to-Robot Pose Estimation from a Single Image](/publication/2020-05_camera-robot-pose-estimation-single-image) Timothy E. Lee, [Jonathan Tremblay](/person/jonathan-tremblay), Thang To, Jia Cheng, Terry Mosier, Oliver Kroemer, Dieter Fox, [Stan Birchfield](/person/stan-birchfield) ICRA 2020 [Toward Sim-to-Real Directional Semantic Grasping](/publication/2020-05_toward-sim-real-directional-semantic-grasping) Shariq Iqbal, [Jonathan Tremblay](/person/jonathan-tremblay), Thang To, Jia Cheng, Erik Leitch, Andy Campbell, Kirby Leung, Duncan McKay, [Stan Birchfield](/person/stan-birchfield) ICRA 2020 [SymGAN: Orientation Estimation without Annotation for Symmetric Objects](/publication/2020-03_symgan-orientation-estimation-without-annotation-symmetric-objects) Phil Ammirato, [Jonathan Tremblay](/person/jonathan-tremblay), [Ming-Yu Liu](/person/ming-yu-liu), Alexander Berg, Dieter Fox [WACV](https://wacv20.wacv.net/) ### 2019 [PAMTRI: Pose-Aware Multi-Task Learning for Vehicle Re-Identification Using Highly Randomized Synthetic Data](/publication/2019-10_pamtri-pose-aware-multi-task-learning-vehicle-re-identification-using-highly) Zheng Tang, Milind Naphade, [Stan Birchfield](/person/stan-birchfield), [Jonathan Tremblay](/person/jonathan-tremblay), William Hodge, Ratnesh Kumar, Shuo Wong, [Xiaodong Yang](/person/xiaodong-yang) [ICCV 2019](http://iccv2019.thecvf.com/) [Few-Shot Viewpoint Estimation](/publication/2019-09_few-shot-viewpoint-estimation) Hung-Yu Tseng, [Shalini De Mello](/person/shalini-de-mello), [Jonathan Tremblay](/person/jonathan-tremblay), [Sifei Liu](/person/sifei-liu), [Stan Birchfield](/person/stan-birchfield), Ming-Hsuan Yang, [Jan Kautz](/person/jan-kautz) [British Machine Vision Conference (BMVC) 2019](https://bmvc2019.org/programme/) ### 2018 [Deep Object Pose Estimation for Semantic Robotic Grasping of Household Objects](/publication/2018-09_deep-object-pose-estimation-semantic-robotic-grasping-household-objects) [Jonathan Tremblay](/person/jonathan-tremblay), Thang To, Bala Sundaralingam, Yu Xiang, Dieter Fox, [Stan Birchfield](/person/stan-birchfield) [Conference on Robot Learning (CoRL) 2018](http://www.robot-learning.org/) [Falling Things: A Synthetic Dataset for 3D Object Detection and Pose Estimation](/index.php/publication/2018-06_falling-things-synthetic-dataset-3d-object-detection-and-pose-estimation) [Jonathan Tremblay](/index.php/person/jonathan-tremblay), Thang To, [Stan Birchfield](/index.php/person/stan-birchfield) [CVPR 2018 Workshop on Real World Challenges and New Benchmarks for Deep Learnin…](https://sites.google.com/view/cvpr2018-robotic-vision) [Synthetically Trained Neural Networks for Learning Human-Readable Plans from Real-World Demonstrations ](/publication/2018-05_synthetically-trained-neural-networks-learning-human-readable-plans-real-world) [Jonathan Tremblay](/person/jonathan-tremblay), Thang To, Artem Molchanov, [Stephen Tyree](/person/stephen-tyree), [Jan Kautz](/person/jan-kautz), [Stan Birchfield](/person/stan-birchfield) [IEEE International Conference on Robotics and Automation (ICRA) 2018](https://icra2018.org/) [Training Deep Networks with Synthetic Data: Bridging the Reality Gap by Domain Randomization](/publication/2018-04_training-deep-networks-synthetic-data-bridging-reality-gap-domain-randomization) [Jonathan Tremblay](/person/jonathan-tremblay), Aayush Prakash, David Acuna, Mark Brophy, Varun Jampani, Cem Anil, Thang To, Eric Cameracci, Shaad Boochoon, [Stan Birchfield](/person/stan-birchfield) [CVPR 2018 Workshop on Autonomous Driving](http://www.wad.ai/)