Publications

Our publications provide insight into some of our leading-edge research.

Filters

Filters

Publication Year

Research Areas

Events

2026

NVIDIA OmniDreams: Real-Time Generative World Model for Closed-Loop Autonomous Vehicle Simulation
Aarti Basant, Amlan Kar, Despoina Paschalidou, Fangyin Wei, Francesco Ferroni, Guillermo Garcia Cobo, Haithem Turki, Huan Ling, Jaewoo Seo, James Lucas, Jay Zhangjie Wu, Jialiang Wang, Jonathan Lorraine, Jun Gao, Kai He, Katarina Tothova, Kevin Xie, Michał Tyszkiewicz, Qi Wu, Riccardo de Lutio, Ruilong Li, Sanja Fidler, Seung Wook Kim, Tianchang Shen, Tianshi Cao, Tobias Pfaff, William Lew, Xindi Wu, Xuanchi Ren, Yifan Lu, Yuxuan Zhang, Zan Gojcic, Zian Wang

2025

Beyond Behavior Cloning in Autonomous Driving: a Survey of Closed-Loop Training Techniques
Peter Karkus, Maximilian Igl, Yuxiao Chen, Kashyap Chitta, Jef Packer, Bertrand Douillard, Thomas Tian, Alexander Naumann, Guillermo Garcia-Cobo, Shuhan Tan, Alperen Degirmenci, Alexander Popov, Nikolai Smolyanskiy, Urs Muller, Boris Ivanovic, Marco Pavone
Alpamayo 1: Bridging Reasoning and Action Prediction for Generalizable Autonomous Driving in the Long Tail
Marco Pavone, Many other contributors found on Page 33
World Simulation With Video Foundation Models for Physical AI
Ming-Yu Liu,
Sim2Val: Leveraging Correlation Across Test Platforms for Variance-Reduced Metric Estimation
Rachel Luo, Heng Yang, Michael Watson, Apoorva Sharma, Sushant Veer, Edward Schmerling, Marco Pavone
GEN3C: 3D-Informed World-Consistent Video Generation with Precise Camera Control
Xuanchi Ren, Tianchang Shen, Jiahui Huang, Huan Ling, Yifan Lu, Merlin Nimier-David, Thomas Müller, Alex Keller, Sanja Fidler, Jun Gao
Cosmos Transfer 1: World-to-World Transfer with Adaptive Multi-Control for Physical AI
Ming-Yu Liu
Cosmos-Reason 1: From Physical AI Common Sense to Embodied Decisions
Tsung-Yi Lin, Ming-Yu Liu
NVIDIA Isaac GR00T N1: An Open Foundation Model for Humanoid Robots
Yuke Zhu, Linxi "Jim" Fan, NVIDIA GEAR Team
Cosmos World Foundation Model Platform for Physical AI
Ming-Yu Liu, Many other contributors at https://d1qx31qr3h6wln.cloudfront.net/publications/NVIDIA%20Cosmos_4.pdf, Jing Zhang

2024

Mitigating Covariate Shift in Imitation Learning for Autonomous Vehicles Using Latent Space Generative World Models
Alexander Popov, Alperen Degirmenci, David Wehr, Shashank Hegde , Ryan Oldja, Alexey Kamenev, Bertrand Douillard, David Nistér, Urs Muller, Ruchi Bhargava, Stan Birchfield, Nikolai Smolyanskiy
Real-Time Anomaly Detection and Reactive Planning with Large Language Models
Rohan Sinha, Amine Elhafsi, Christopher Agia, Matthew Foutter, Edward Schmerling, Marco Pavone

2023

Learning for CasADi: Data-driven Models in Numerical Optimization
Tim Salzmann, Jon Arrizabalaga, Joel Andersson, Marco Pavone, Markus Ryll
PAC-Bayes Generalization Certificates for Learned Inductive Conformal Prediction
Apoorva Sharma, Sushant Veer, Asher Hancock, Heng Yang, Marco Pavone, Anirudha Majumdar
Augmenting Lane Perception and Topology Understanding with Standard Definition Navigation Maps
Katie Z Luo, Xinshuo Weng, Yan Wang, Shuang Wu, Jie Li, Kilian Q Weinberger, Yue Wang, Marco Pavone
Interpretable Trajectory Prediction for Autonomous Vehicles via Counterfactual Responsibility
Kai-Chieh Hsu, Karen Leung, Yuxiao Chen, Jaime Fernandez Fisac, Marco Pavone
trajdata: A Unified Interface to Multiple Human Trajectory Datasets
Boris Ivanovic, Guanyu Song, Igor Gilitschenski, Marco Pavone
Language Conditioned Traffic Generation
Shuhan Tan, Boris Ivanovic, Xinshuo Weng, Marco Pavone, Philipp Krähenbühl
Task-Aware Risk Estimation of Perception Failures for Autonomous Vehicles
Pasquale Antonante, Sushant Veer, Karen Leung, Xinshuo Weng, Luca Carlone, Marco Pavone
Refining Obstacle Perception Safety Zones via Maneuver-Based Decomposition
Sever Topan, Yuxiao Chen, Ed Schmerling, Karen Leung, Jonas Nilsson, Michael Cox, Marco Pavone
Learning Autonomous Vehicle Safety Concepts from Demonstrations
Karen Leung, Sushant Veer, Edward Schmerling, Marco Pavone
Tree-structured Policy Planning with Learned Behavior Models
Yuxiao Chen, Peter Karkus, Boris Ivanovic, Xinshuo Weng, Marco Pavone
Receding Horizon Planning with Rule Hierarchies for Autonomous Vehicles
Sushant Veer, Karen Leung, Ryan Cosner, Yuxiao Chen, Marco Pavone
Planning with Occluded Traffic Agents using Bi-Level Variational Occlusion Models
Filippos Christianos, Peter Karkus, Boris Ivanovic, Stefano V. Albrecht, Marco Pavone
Guided Conditional Diffusion for Controllable Traffic Simulation
Ziyuan Zhong, Davis Rempe, Danfei Xu, Yuxiao Chen, Sushant Veer, Gerry Che, Baishakhi Ray, Marco Pavone
Expanding the Deployment Envelope of Behavior Prediction via Adaptive Meta-Learning
Boris Ivanovic, James Harrison, Marco Pavone
BITS: Bi-level Imitation for Traffic Simulation
Danfei Xu, Yuxiao Chen, Boris Ivanovic, Marco Pavone
Semantic Anomaly Detection with Large Language Models
Amine Elhafsi, Rohan Sinha, Christopher Agia, Edward Schmerling, Issa Nesnas, Marco Pavone
Bayesian Reparameterization of Reward-Conditioned Reinforcement Learning with Energy-based Models
Wenhao Ding, Tong Che, Ding Zhao, Marco Pavone
Task-Aware Risk Estimation of Perception Failures for Autonomous Vehicles
Pasquale Antonante, Sushant Veer, Karen Leung, Xinshuo Weng, Luca Carlone, Marco Pavone
FreeNeRF: Improving Few-shot Neural Rendering with Free Frequency Regularization
Heng Yang, Marco Pavone

2022

Task-Relevant Failure Detection for Trajectory Predictors in Autonomous Vehicles
Alec Farid, Sushant Veer, Boris Ivanovic, Karen Leung, Marco Pavone