Publications

Our publications provide insight into some of our leading-edge research.

Filters

Filters

Publication Year

Research Areas

Events

2026

3D-GENERALIST: Vision-Language-Action Models for Crafting 3D Worlds
Fan-Yun Sun, Shengguang Wu, Christian Jacobsen, Thomas Yim, Haoming Zou, Alex Zook, Shangru Li, Yu-Hsin Chou, Ethem Can, Xunlei Wu, Clemens Eppner, Valts Blukis, Jonathan Tremblay, Jiajun Wu, Stan Birchfield, Nick Haber
Demystifying Data-Driven Probabilistic Medium-Range Weather Forecasting
Jean Kossaifi, Nikola Kovachki, Morteza Mardani, Daniel Leibovici, Suman Ravuri, Ira Shokar, Edoardo Calvello, Mohammad Shoaib Abbas, Peter Harrington, Ashay Subramaniam, Noah Brenowitz, Boris Bonev, Wonmin Byeon, Karsten Kreis, Dale Durran, Arash Vahdat, Mike Pritchard, Jan Kautz
HealDA: Highlighting the importance of initial errors in end-to-end AI weather forecasts
Aayush Gupta, Akshay Subramaniam, Michael S. Pritchard, Karthik Kashinath, Sergey Frolov, Kelsey Lieberman, Christopher Miller, Nicholas Silverman, Noah D. Brenowitz
Learning Accurate Storm-Scale Evolution from Observations
Jaideep Pathak, Mohammad Shoaib Abbas, Peter Harrington, Zeyuan Hu, Noah Brenowitz, Suman Ravuri, Alberto Carpentieri, Jussi Leinonen, Corey Adams, Oliver Hennigh, Nicholas Geneva, Dale Durran, Mike Pritchard

2025

Play4D: Accelerated and Interactive Free-viewpoint Video Streaming for Virtual Reality and Light Field Displays
Jonghyun Kim, Michael Stengel, Amrita Mazumdar, Tianye Li, Cheng Sun, David Luebke, Shalini De Mello
Beyond Behavior Cloning in Autonomous Driving: a Survey of Closed-Loop Training Techniques
Peter Karkus, Maximilian Igl, Yuxiao Chen, Kashyap Chitta, Jef Packer, Bertrand Douillard, Thomas Tian, Alexander Naumann, Guillermo Garcia-Cobo, Shuhan Tan, Alperen Degirmenci, Alexander Popov, Nikolai Smolyanskiy, Urs Muller, Boris Ivanovic, Marco Pavone
Policy Optimized Text-to-Image Pipeline Design
Uri Gadot, Rinon Gal, Yftah Zisser, Gal Chechik, Shie Mannor
Elucidated Rolling Diffusion Models for Probabilistic Weather Forecasting
Salva Rühling Cachay, Miika Aittala, Karsten Kreis, Noah Brenowitz, Arash Vahdat, Morteza Mardani, Rose Yu
Align Your Flow: Scaling Continuous-Time Flow Map Distillation
Amirmojtaba Sabour, Sanja Fidler, Karsten Kreis
ThinkAct: Vision-Language-Action Reasoning via Reinforced Visual Latent Planning
Chi-Pin Huang, Yueh-Hua Wu, Min-Hung Chen, Frank Wang, Fred Yang
Generalizable Domain Adaptation for Sim-and-Real Policy Co-Training
Shuo Cheng, Liqian Ma, Zhenyang Chen, Ajay Mandlekar, Caelan Garrett, Danfei Xu
RaySt3R: Predicting Novel Depth Maps for Zero-Shot Object Completion
Bardienus P. Duisterhof, Jan Oberst, Bowen Wen, Stan Birchfield, Deva Ramanan, Jeffrey Ichnowski
Attention on the Sphere
Boris Bonev, Max Rietmann, Andrea Paris, Alberto Carpentieri, Thorsten Kurth
Seeing What Matters: Generalizable AI-generated Video Detection with Forensic-Oriented Augmentation
Riccardo Corvi, Davide Cozzolino, Ekta Prashnani, Shalini De Mello, Koki Nagano, Luisa Verdoliva
Augmenting Simulated Noisy Quantum Data Collection by Orders of Magnitude Using Pre-Trajectory Sampling with Batched Execution
Taylor Patti, Thien Nguyen, Justin Lietz, Alex McCaskey, Brucek Khailany
Data-Driven Loss Functions for Inference-Time Optimization in Text-to-Image
Sapir Yflah, Yuval Atzmon, Gal Chechik
ScheduleStream: Temporal Planning with Samplers for GPU-Accelerated Multi-Arm Task and Motion Planning & Scheduling
Caelan Garrett, Fabio Ramos
Do What You Say: Steering Vision-Language-Action Models via Runtime Reasoning-Action Alignment Verification
Yilin Wu, Anqi Li, Tucker Hermans, Fabio Ramos, Andrea Bajcs, Claudia Pérez D’Arpino
Alpamayo 1: Bridging Reasoning and Action Prediction for Generalizable Autonomous Driving in the Long Tail
Marco Pavone, Many other contributors found on Page 33
The Reality Gap in Robotics: Challenges, Solutions, and Best Practices
Elie Aljalbout, Jiaxu Xing, Angel Romero, Iretiayo Akinola, Caelan Garrett, Eric Heiden, Abhishek Gupta, Tucker Hermans, Yashraj Narang, Dieter Fox, Davide Scaramuzza, Fabio Ramos
Task-Oriented Human Grasp Synthesis via Context- and Task-Aware Diffusers
An-Lun Liu, Yu-Wei Chao, Yi-Ting Chen
ACGD: Visual Multitask Policy Learning with Asymmetric Critic Guided Distillation
Krishnan Srinivasan, Jie Xu, Henry Ang, Eric Heiden, Dieter Fox, Jeannette Bohg, Animesh Garg
SALAD: Self-Adaptive Link Adaptation
Reinhard Wiesmayr, Lorenzo Maggi, Sebastian Cammerer, Jakob Hoydis, Fayçal Aït Aoudia , Alex Keller
Pedestrian Collision Avoidance in Hemianopia during Natural Walking in Immersive Virtual Reality
Jonathan K. Doyon, Sujin Kim, Alex D. Hwang, Jae-Hyun Jung
Isaac Lab: A GPU Accelerated Simulation Framework For Multi-Modal Robot Learning
Mayank Mittal, Kelly Guo, Gavriel State, Spencer Huang
World Simulation With Video Foundation Models for Physical AI
Ming-Yu Liu,
VT-Refine: Learning Bimanual Assembly with Visuo-Tactile Feedback via Simulation Fine-Tuning
Binghao Huang, Jie Xu, Iretiayo Akinola, Wei Yang, Balakumar Sundaralingam, Rowland O'Flaherty, Dieter Fox, Xiaolong Wang, Arsalan Mousavian, Yu-Wei Chao, Yunzhu Li
Dexplore: Scalable Neural Control for Dexterous Manipulation from Reference-Scoped Exploration
Sirui Xu, Yu-Wei Chao, Liuyu Bian, Arsalan Mousavian, Yu-Xiong Wang, Liang-Yan Gui, Wei Yang
Neural Robot Dynamics
Jie Xu, Eric Heiden, Iretiayo Akinola, Dieter Fox, Miles Macklin, Yashraj Narang
Sim2Val: Leveraging Correlation Across Test Platforms for Variance-Reduced Metric Estimation
Rachel Luo, Heng Yang, Michael Watson, Apoorva Sharma, Sushant Veer, Edward Schmerling, Marco Pavone
Learning to Plan & Schedule with Reinforcement-Learned Bimanual Robot Skills
Weikang Wan, Fabio Ramos, Xuning Yang, Caelan Garrett
ReinforceGen: Hybrid Skill Policies with Automated Data Generation and Reinforcement Learning
Zihan Zhou, Animesh Garg, Ajay Mandlekar, Caelan Garrett