Publications

Producing and Leveraging Online Map Uncertainty in Trajectory Prediction

CVPR 2024 (Best Paper Award Finalist)

High-definition (HD) maps have played an integral role in the development of modern autonomous vehicle (AV) stacks, albeit with high …
PDF
Producing and Leveraging Online Map Uncertainty in Trajectory Prediction

Parallelized Spatiotemporal Binding

ICML 2024

While modern best practices advocate for scalable architectures that support long-range interactions, object-centric models are yet to …
PDF
Parallelized Spatiotemporal Binding

InstantSplat: Unbounded Sparse-view Pose-free Gaussian Splatting in 40 Seconds

CVPR NRI 2024

While novel view synthesis (NVS) has made substantial progress in 3D computer vision, it typically requires an initial estimation of …
PDF Project
InstantSplat: Unbounded Sparse-view Pose-free Gaussian Splatting in 40 Seconds

Driving Everywhere with Large Language Model Policy Adaptation

CVPR 2024

Adapting driving behavior to new environments, customs, and laws is a long-standing problem in autonomous driving, precluding the …
PDF Project
Driving Everywhere with Large Language Model Policy Adaptation

RealGen: Retrieval Augmented Generation for Controllable Traffic Scenarios

arXiv

Simulation plays a crucial role in the development of autonomous vehicles (AVs) due to the potential risks associated with real-world …
PDF Project
RealGen: Retrieval Augmented Generation for Controllable Traffic Scenarios

Dolphins: Multimodal Language Model for Driving

arXiv

The quest for fully autonomous vehicles (AVs) capable of navigating complex real-world scenarios with human-like understanding and …
PDF Project
Dolphins: Multimodal Language Model for Driving

A Language Agent for Autonomous Driving

arXiv

Human-level driving is an ultimate goal of autonomous driving. Conventional approaches formulate autonomous driving as a …
PDF Project
A Language Agent for Autonomous Driving

Categorical Traffic Transformer: Interpretable and Diverse Behavior Prediction with Tokenized Latent

arXiv

Adept traffic models are critical to both planning and closed-loop simulation for autonomous vehicles (AV), and key design objectives …
PDF
Categorical Traffic Transformer: Interpretable and Diverse Behavior Prediction with Tokenized Latent

EmerNeRF: Emergent Spatial-Temporal Scene Decomposition via Self-Supervision

arXiv

We present EmerNeRF, a simple yet powerful approach for learning spatial-temporal representations of dynamic driving scenes. Grounded …
PDF Code Dataset Project
EmerNeRF: Emergent Spatial-Temporal Scene Decomposition via Self-Supervision

Partial-View Object View Synthesis via Filtered Inversion

3DV 2024 (Spotlight)

We propose Filtering Inversion (FINV), a learning framework and optimization process that predicts a renderable 3D object …
PDF Code Project
Partial-View Object View Synthesis via Filtered Inversion

trajdata: A Unified Interface to Multiple Human Trajectory Datasets

NeurIPS 2023 Datasets and Benchmarks Track

The field of trajectory forecasting has grown significantly in recent years, partially owing to the release of numerous large-scale, …
PDF Code
trajdata: A Unified Interface to Multiple Human Trajectory Datasets

PAC-Bayes Generalization Certificates for Learned Inductive Conformal Prediction

NeurIPS 2023

Inductive Conformal Prediction (ICP) provides a practical and effective approach for equipping deep learning models with uncertainty …
PDF
PAC-Bayes Generalization Certificates for Learned Inductive Conformal Prediction

Language-Guided Traffic Simulation via Scene-Level Diffusion

CoRL 2023 (Oral)

Realistic and controllable traffic simulation is a core capability that is necessary to accelerate autonomous vehicle (AV) development. …
PDF
Language-Guided Traffic Simulation via Scene-Level Diffusion

Language Conditioned Traffic Generation

CoRL 2023

Simulation forms the backbone of modern self-driving development. Simulators help develop, test, and improve driving systems without …
PDF Code Project Video
Language Conditioned Traffic Generation

Multi-Predictor Fusion: Combining Learning-based and Rule-based Trajectory Predictors

CoRL 2023

Trajectory prediction modules are key enablers for safe and efficient planning of autonomous vehicles (AVs), particularly in highly …
PDF
Multi-Predictor Fusion: Combining Learning-based and Rule-based Trajectory Predictors

DTPP: Differentiable Joint Conditional Prediction and Cost Evaluation for Tree Policy Planning in Autonomous Driving

arXiv

Motion prediction and cost evaluation are vital components in the decision-making system of autonomous vehicles. However, existing …
PDF
DTPP: Differentiable Joint Conditional Prediction and Cost Evaluation for Tree Policy Planning in Autonomous Driving

Reinforcement Learning with Human Feedback for Realistic Traffic Simulation

arXiv

In light of the challenges and costs of real-world testing, autonomous vehicle developers often rely on testing in simulation for the …
PDF
Reinforcement Learning with Human Feedback for Realistic Traffic Simulation

Interactive Joint Planning for Autonomous Vehicles

arXiv

In highly interactive driving scenarios, the actions of one agent greatly influences those of its neighbors. Planning safe motions for …
PDF
Interactive Joint Planning for Autonomous Vehicles

Augmenting Lane Perception and Topology Understanding with Standard Definition Navigation Maps

arXiv

Autonomous driving has traditionally relied heavily on costly and labor-intensive High Definition (HD) maps, hindering scalability. In …
PDF
Augmenting Lane Perception and Topology Understanding with Standard Definition Navigation Maps

Verification and Synthesis of Robust Control Barrier Functions: Multilevel Polynomial Optimization and Semidefinite Relaxation

CDC 2023

We study the problem of verification and synthesis of robust control barrier functions (CBF) for control-affine polynomial systems with …
PDF
Verification and Synthesis of Robust Control Barrier Functions: Multilevel Polynomial Optimization and Semidefinite Relaxation

Interpretable Trajectory Prediction for Autonomous Vehicles via Counterfactual Responsibility

IROS 2023

The ability to anticipate surrounding agents’ behaviors is critical to enable safe and seamless autonomous vehicles (AVs). While …
Project
Interpretable Trajectory Prediction for Autonomous Vehicles via Counterfactual Responsibility

Bayesian Reparameterization of Reward-Conditioned Reinforcement Learning with Energy-based Models

ICML 2023

Recently, reward-conditioned reinforcement learning (RCRL) has gained popularity due to its simplicity, flexibility, and off-policy …
PDF
Bayesian Reparameterization of Reward-Conditioned Reinforcement Learning with Energy-based Models

Task-Aware Risk Estimation of Perception Failures for Autonomous Vehicles

RSS 2023

Safety and performance are key enablers for autonomous driving: on the one hand we want our autonomous vehicles (AVs) to be safe, while …
PDF Code
Task-Aware Risk Estimation of Perception Failures for Autonomous Vehicles

Refining Obstacle Perception Safety Zones via Maneuver-Based Decomposition

IV 2023

A critical task for developing safe autonomous driving stacks is to determine whether an obstacle is safety-critical, i.e., poses an …
PDF
Refining Obstacle Perception Safety Zones via Maneuver-Based Decomposition

Object Pose Estimation with Statistical Guarantees: Conformal Keypoint Detection and Geometric Uncertainty Propagation

CVPR 2023 (Highlight)

The two-stage object pose estimation paradigm first detects semantic keypoints on the image and then estimates the 6D pose by …
PDF
Object Pose Estimation with Statistical Guarantees: Conformal Keypoint Detection and Geometric Uncertainty Propagation

FreeNeRF: Improving Few-shot Neural Rendering with Free Frequency Regularization

CVPR 2023

Novel view synthesis with sparse inputs is a challenging problem for neural radiance fields (NeRF). Recent efforts alleviate this …
PDF Code Project
FreeNeRF: Improving Few-shot Neural Rendering with Free Frequency Regularization

Learning Autonomous Vehicle Safety Concepts from Demonstrations

ACC 2023

Evaluating the safety of an autonomous vehicle (AV) depends on the behavior of surrounding agents which can be heavily influenced by …
PDF
Learning Autonomous Vehicle Safety Concepts from Demonstrations

Tree-structured Policy Planning with Learned Behavior Models

ICRA 2023

Autonomous vehicles (AVs) need to reason about the multimodal behavior of neighboring agents while planning their own motion. Many …
PDF
Tree-structured Policy Planning with Learned Behavior Models

Receding Horizon Planning with Rule Hierarchies for Autonomous Vehicles

ICRA 2023

Autonomous vehicles must often contend with conflicting planning requirements, e.g., safety and comfort could be at odds with each …
PDF
Receding Horizon Planning with Rule Hierarchies for Autonomous Vehicles

Planning with Occluded Traffic Agents using Bi-Level Variational Occlusion Models

ICRA 2023

Reasoning with occluded traffic agents is a significant open challenge for planning for autonomous vehicles. Recent deep learning …
PDF
Planning with Occluded Traffic Agents using Bi-Level Variational Occlusion Models

Guided Conditional Diffusion for Controllable Traffic Simulation

ICRA 2023

Controllable and realistic traffic simulation is critical for developing and verifying autonomous vehicles. Typical heuristic-based …
PDF Project
Guided Conditional Diffusion for Controllable Traffic Simulation

Expanding the Deployment Envelope of Behavior Prediction via Adaptive Meta-Learning

ICRA 2023

Learning-based behavior prediction methods are increasingly being deployed in real-world autonomous systems, e.g., in fleets of …
PDF Code
Expanding the Deployment Envelope of Behavior Prediction via Adaptive Meta-Learning

BITS: Bi-level Imitation for Traffic Simulation

ICRA 2023

Simulation is the key to scaling up validation and verification for robotic systems such as autonomous vehicles. Despite advances in …
PDF
BITS: Bi-level Imitation for Traffic Simulation

Robust and Controllable Object-Centric Learning through Energy-based Models

ICLR 2023

Humans are remarkably good at understanding and reasoning about complex visual scenes. The capability to decompose low-level …
PDF
Robust and Controllable Object-Centric Learning through Energy-based Models

Robust Trajectory Prediction against Adversarial Attacks

CoRL 2022 (Oral)

Trajectory prediction using deep neural networks (DNNs) is an essential component of autonomous driving (AD) systems. However, these …
PDF Code Project
Robust Trajectory Prediction against Adversarial Attacks

Task-Relevant Failure Detection for Trajectory Predictors in Autonomous Vehicles

CoRL 2022

In modern autonomy stacks, prediction modules are paramount to planning motions in the presence of other mobile agents. However, …
PDF
Task-Relevant Failure Detection for Trajectory Predictors in Autonomous Vehicles

DiffStack: A Differentiable and Modular Control Stack for Autonomous Vehicles

CoRL 2022

Autonomous vehicle (AV) stacks are typically built in a modular fashion, with explicit components performing detection, tracking, …
PDF Code Project
DiffStack: A Differentiable and Modular Control Stack for Autonomous Vehicles

AdvDO: Realistic Adversarial Attacks for Trajectory Prediction

ECCV 2022

Trajectory prediction is essential for autonomous vehicles (AVs) to plan correct and safe driving behaviors. While many prior works aim …
PDF
AdvDO: Realistic Adversarial Attacks for Trajectory Prediction

Towards Data-Driven Synthesis of Autonomous Vehicle Safety Concepts

arXiv

As safety-critical autonomous vehicles (AVs) will soon become pervasive in our society, a number of safety concepts for trusted AV …
PDF
Towards Data-Driven Synthesis of Autonomous Vehicle Safety Concepts

Whose Track Is It Anyway? Improving Robustness to Tracking Errors with Affinity-Based Prediction

CVPR 2022

Multi-agent trajectory prediction is critical for planning and decision-making in human-interactive autonomous systems, such as …
PDF Project
Whose Track Is It Anyway? Improving Robustness to Tracking Errors with Affinity-Based Prediction

ScePT: Scene-consistent, Policy-based Trajectory Predictions for Planning

CVPR 2022

Trajectory prediction is a critical functionality of autonomous systems that share environments with uncontrolled agents, one prominent …
PDF Code
ScePT: Scene-consistent, Policy-based Trajectory Predictions for Planning

Propagating State Uncertainty Through Trajectory Forecasting

ICRA 2022

Uncertainty pervades through the modern robotic autonomy stack, with nearly every component (e.g., sensors, detection, classification, …
PDF Video
Propagating State Uncertainty Through Trajectory Forecasting

Interaction-Dynamics-Aware Perception Zones for Obstacle Detection Safety Evaluation

IV 2022

To enable safe autonomous vehicle (AV) operations, it is critical that an AV’s obstacle detection module can reliably detect …
PDF
Interaction-Dynamics-Aware Perception Zones for Obstacle Detection Safety Evaluation

MTP: Multi-Hypothesis Tracking and Prediction for Reduced Error Propagation

IV 2022

Recently, there has been tremendous progress in developing each individual module of the standard perception-planning robot autonomy …
PDF Project Video
MTP: Multi-Hypothesis Tracking and Prediction for Reduced Error Propagation

Injecting Planning-Awareness into Prediction and Detection Evaluation

IV 2022

Detecting other agents and forecasting their behavior is an integral part of the modern robotic autonomy stack, especially in …
PDF Video
Injecting Planning-Awareness into Prediction and Detection Evaluation

Rethinking Trajectory Forecasting Evaluation

arXiv

Forecasting the behavior of other agents is an integral part of the modern robotic autonomy stack, especially in safety-critical …
PDF
Rethinking Trajectory Forecasting Evaluation