Generalist Embodied Agent Research | NVIDIA
NVIDIA GEAR

Publications

Highlights

Eureka: Human-Level Reward Design via Coding Large Language Models
  • Jason Ma ✉, William Liang, Guanzhi Wang, De-An Huang, Osbert Bastani, Dinesh Jayaraman, Yuke Zhu, Linxi "Jim" Fan ✉†, Anima Anandkumar †

    • International Conference on Learning Representations (ICLR), May 2024

    Top 10 NVIDIA Research Highlights of 2023

  • project
  • paper
  • code
  • announcement
    • Voyager: An Open-Ended Embodied Agent with Large Language Models
  • Guanzhi Wang ✉, Yuqi Xie, Yunfan Jiang*, Ajay Mandlekar*, Chaowei Xiao, Yuke Zhu, Linxi "Jim" Fan ✉†, Anima Anandkumar †

    • Transactions on Machine Learning Research (TMLR), February 2024

  • project
  • paper
  • code
  • announcement
    • MimicPlay: Long-Horizon Imitation Learning by Watching Human Play
  • Chen Wang, Linxi "Jim" Fan, Jiankai Sun, Ruohan Zhang, Li Fei-Fei, Danfei Xu, Yuke Zhu*, Anima Anandkumar*

    • Conference on Robot Learning (CoRL), November 2023

    Best Paper Award Finalist

    Best Systems Paper Award Finalist

  • project
  • paper
  • code
  • announcement
    • VIMA: General Robot Manipulation with Multimodal Prompts
  • Yunfan Jiang, Agrim Gupta*, Zichen "Charles" Zhang*, Guanzhi Wang*, Yongqiang Dou, Yanjun Chen, Li Fei-Fei, Anima Anandkumar, Yuke Zhu †, Linxi "Jim" Fan †

    • International Conference on Machine Learning (ICML), July 2023

  • project
  • paper
  • code
  • models
  • benchmark
  • dataset
    • MineDojo: Building Open-Ended Embodied Agents with Internet-Scale Knowledge
  • Linxi "Jim" Fan ✉, Guanzhi Wang ✉*, Yunfan Jiang ✉*, Ajay Mandlekar, Yuncong Yang, Haoyi Zhu, Andrew Tang, De-An Huang, Yuke Zhu †, Anima Anandkumar †

    • Neural Information Processing Systems (NeurIPS), November 2022

    Outstanding Paper Award

  • project
  • paper
  • code
  • dataset
  • announcement

    • All Publications

    Eureka: Human-Level Reward Design via Coding Large Language Models
  • Jason Ma ✉, William Liang, Guanzhi Wang, De-An Huang, Osbert Bastani, Dinesh Jayaraman, Yuke Zhu, Linxi "Jim" Fan ✉†, Anima Anandkumar †

    • International Conference on Learning Representations (ICLR), May 2024

    Top 10 NVIDIA Research Highlights of 2023

  • project
  • paper
  • code
  • announcement
    • Voyager: An Open-Ended Embodied Agent with Large Language Models
  • Guanzhi Wang ✉, Yuqi Xie, Yunfan Jiang*, Ajay Mandlekar*, Chaowei Xiao, Yuke Zhu, Linxi "Jim" Fan ✉†, Anima Anandkumar †

    • Transactions on Machine Learning Research (TMLR), February 2024

  • project
  • paper
  • code
  • announcement
    • Blog Image
    Prismer: A Vision-Language Model with Multi-Task Experts
  • Shikun Liu, Linxi "Jim" Fan, Edward Johns, Zhiding Yu, Chaowei Xiao, Anima Anandkumar

    • Transactions on Machine Learning Research (TMLR), 2024

  • project
  • paper
  • code
  • demo
    • Cross-Episodic Curriculum for Transformer Agents
  • Lucy Xiaoyang Shi*, Yunfan Jiang*, Jake Grigsby, Linxi "Jim" Fan †, Yuke Zhu †

    • Neural Information Processing Systems (NeurIPS), December 2023

  • project
  • paper
  • code
    • Blog Image
    MimicGen: A Data Generation System for Scalable Robot Learning using Human Demonstrations
  • Ajay Mandlekar, Soroush Nasiriany*, Bowen Wen*, Iretiayo Akinola, Yashraj Narang, Linxi "Jim" Fan, Yuke Zhu, Dieter Fox

    • Conference on Robot Learning (CoRL), November 2023

  • project
  • paper
  • code
  • announcement
    • MimicPlay: Long-Horizon Imitation Learning by Watching Human Play
  • Chen Wang, Linxi "Jim" Fan, Jiankai Sun, Ruohan Zhang, Li Fei-Fei, Danfei Xu, Yuke Zhu*, Anima Anandkumar*

    • Conference on Robot Learning (CoRL), November 2023

    Best Paper Award Finalist

    Best Systems Paper Award Finalist

  • project
  • paper
  • code
  • announcement
    • VIMA: General Robot Manipulation with Multimodal Prompts
  • Yunfan Jiang, Agrim Gupta*, Zichen "Charles" Zhang*, Guanzhi Wang*, Yongqiang Dou, Yanjun Chen, Li Fei-Fei, Anima Anandkumar, Yuke Zhu †, Linxi "Jim" Fan †

    • International Conference on Machine Learning (ICML), July 2023

  • project
  • paper
  • code
  • models
  • benchmark
  • dataset
    • MineDojo: Building Open-Ended Embodied Agents with Internet-Scale Knowledge
  • Linxi "Jim" Fan ✉, Guanzhi Wang ✉*, Yunfan Jiang ✉*, Ajay Mandlekar, Yuncong Yang, Haoyi Zhu, Andrew Tang, De-An Huang, Yuke Zhu †, Anima Anandkumar †

    • Neural Information Processing Systems (NeurIPS), November 2022

    Outstanding Paper Award

  • project
  • paper
  • code
  • dataset
  • announcement
    • Blog Image
    Pre-Trained Language Models for Interactive Decision-Making
  • Shuang Li, Xavier Puig, Chris Paxton, Yilun Du, Clinton Wang, Linxi "Jim" Fan, Tao Chen, De-An Huang, Ekin Akyürek, Anima Anandkumar, Jacob Andreas, Igor Mordatch, Antonio Torralba, Yuke Zhu

    • Neural Information Processing Systems (NeurIPS), November 2022

  • project
  • paper
  • code
    • Blog Image
    ACID: Action-Conditional Implicit Visual Dynamics for Deformable Object Manipulation
  • Bokui Shen*, Zhenyu Jiang*, Christopher Choy, Silvio Savarese, Leonidas J. Guibas, Anima Anandkumar, Yuke Zhu

    • Robotics: Science and Systems (RSS), June 2022

    Best Student Paper Award Finalist

  • project
  • paper
  • code
    • Blog Image
    OSCAR: Data-Driven Operational Space Control for Adaptive and Robust Robot Manipulation
  • Josiah Wong, Viktor Makoviychuk, Anima Anandkumar, Yuke Zhu

    • IEEE International Conference on Robotics and Automation (ICRA), May 2022

  • project
  • paper
  • code
    • Blog Image
    SECANT: Self-Expert Cloning for Zero-Shot Generalization of Visual Policies
  • Linxi "Jim" Fan, Guanzhi Wang, De-An Huang, Zhiding Yu, Li Fei-Fei, Yuke Zhu, Animashree Anandkumar

    • International Conference on Machine Learning (ICML), July 2021

  • project
  • paper
  • code
    • Blog Image
    RubiksNet: Learnable 3D-Shift for Efficient Video Action Recognition
  • Linxi "Jim" Fan*, Shyamal Buch*, Guanhzi Wang, Ryan Cao, Yuke Zhu, Juan Carlos Niebles, Li Fei-Fei

    • European Conference on Computer Vision (ECCV), August 2020

  • project
  • paper
  • code
    • Blog Image
    SURREAL: Open-Source Reinforcement Learning Framework and Robot Manipulation Benchmark
  • Linxi "Jim" Fan, Yuke Zhu, Jiren Zhu, Zihua Liu, Orien Zeng, Anchit Gupta, Joan Creus-Costa, Silvio Savarese, Li Fei-Fei

    • Conference of Robotic Learning (CoRL), October 2018

  • project
  • paper
  • code
    • United States
    Copyright © 2024 NVIDIA Corporation