1. [Publications](/index.php/publications) 2. Injecting Planning-Awareness into Prediction and Detection Evaluation # Injecting Planning-Awareness into Prediction and Detection Evaluation ![](/sites/default/files/styles/wide/public/publications/featured.PNG?itok=7N36x_Rs) Detecting other agents and forecasting their behavior is an integral part of the modern robotic autonomy stack, especially in safety-critical scenarios entailing human-robot interaction such as autonomous driving. Due to the importance of these components, there has been a significant amount of interest and research in perception and trajectory forecasting, resulting in a wide variety of approaches. Common to most works, however, is the use of the same few accuracy-based evaluation metrics, e.g., intersection-over-union, displacement error, log-likelihood, etc. While these metrics are informative, they are task-agnostic and outputs that are evaluated as equal can lead to vastly different outcomes in downstream planning and decision making. In this work, we take a step back and critically assess current evaluation metrics, proposing task-aware metrics as a better measure of performance in systems where they are deployed. Experiments on an illustrative simulation as well as real-world autonomous driving data validate that our proposed task-aware metrics are able to account for outcome asymmetry and provide a better estimate of a model’s closed-loop performance. ## Authors [Boris Ivanovic](/index.php/person/boris-ivanovic) [Marco Pavone](/index.php/person/marco-pavone) ## Publication Date Sunday, June 5, 2022 ## Published in [IEEE Intelligent Vehicles Symposium (IV) 2022](https://iv2022.com/) ## Research Area [Artificial Intelligence and Machine Learning ](/index.php/research-area/machine-learning-artificial-intelligence) [Autonomous Vehicles](/index.php/research-area/autonomous-vehicles) [Computer Vision](/index.php/research-area/computer-vision) ## External Links [Paper](https://ieeexplore.ieee.org/document/9827101) ## Copyright This material is posted here with permission of the IEEE. Internal or personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution must be obtained from the IEEE by writing to .