Research Areas

Applied Perception

Associated Publications

2025

Toward Understanding Display Size for FPS Esports Aiming
Arjun Madhusudan, Josef Spjut, Benjamin Watson, Seth Schneider, Ben Boudaoud, Joohwan Kim
Foundations of Digital Games
Modeling Visually-Guided Aim-and-Shoot behavior in First-Person Shoters
June-Seop Yoon, Hee-Seung Moon, Ben Boudaoud, Josef Spjut, Iuri Frosio, Byungjoo Lee, Joohwan Kim
International Journal of Human-Computer Studies
Pushing the Limits? Frame Rate Benefits to Players for up to 500 Hz in First Person Shooter Games
Samin Shahriar Tokey, Ben Boudaoud, Joohwan Kim, Josef Spjut, Mark Claypool
Network and Operating System Support for Digital Audio and Video
CorrFill: Enhancing Faithfulness in Reference-based Inpainting with Correspondence Guidance in Diffusion Models
Kuan-Hung Liu, Cheng-Kun Yang, Min-Hung Chen, Yu-Lun Liu, Yen-Yu Lin
Winter Conference on Applications of Computer Vision (WACV)
Semantic Prompt Learning for Weakly-Supervised Semantic Segmentation
Ci-Siang Lin, Chien-Yi Wang, Frank Wang, Min-Hung Chen
Winter Conference on Applications of Computer Vision (WACV)
Spatio-Temporal Context Prompting for Zero-Shot Action Detection
Wei-Jhe Huang, Min-Hung Chen, Shang-Hong Lai
Winter Conference on Applications of Computer Vision (WACV)

2024

Mitigating Covariate Shift in Imitation Learning for Autonomous Vehicles Using Latent Space Generative World Models
Alexander Popov, Alperen Degirmenci, David Wehr, Shashank Hegde , Ryan Oldja, Alexey Kamenev, Bertrand Douillard, David Nistér, Urs Muller, Ruchi Bhargava, Stan Birchfield, Nikolai Smolyanskiy
arXiv
GenTranslate: Large Language Models are Generative Multilingual Speech and Machine Translators
Yuchen Hu, Chen Chen, Huck Yang, Ruizhe Li, Zhehuai Chen, Eng Siong Chng
ACL 2024
Variable Frame Timing Affects Perception of Smoothness in First-Person Gaming
Devi Klein, Josef Spjut, Ben Boudaoud, Joohwan Kim
IEEE Conference on Games (CoG)
FoundationPose: Unified 6D Pose Estimation and Tracking of Novel Objects
Bowen Wen, Wei Yang, Jan Kautz, Stan Birchfield
CVPR 2024
Do Action Video Game Players Search Faster Than Non-Players?
Zoe (Jing) Xu, Josef Spjut, Ben Boudaoud, Simona Buetti, Alejandro Lleras, Ruth Rosenholtz
Vision Sciences Society
It's Never Too Late: Fusing Acoustic Information into Large Language Models for Automatic Speech Recognition
Chen Chen, Ruizhe Li, Yuchen Hu, Sabato Marco Siniscalchi, Pin-Yu Chen, Ensiong Chng, Huck Yang
ICLR 2024
Large Language Models are Efficient Learners of Noise-Robust Speech Recognition
YuChen Hu, Chen Chen, Huck Yang, Ruizhe Li, Chao Zhang, Pin-Yu Chen, EnSiong Chng
ICLR 2024
Is Less More? Rendering for Esports
Benjamin Watson, Josef Spjut, Joohwan Kim, Byungjoo Lee, Mijin Yoo, Peter Shirley, Rulon Raymond
IEEE Computer Graphics and Applications
Evaluating and Improving Rendered Visual Experiences: Metrics, Compression, Higher Frame Rates & Recoloring
Pontus Ebelin
Doctoral Theses in Mathematical Sciences at Lund University
Estimates of Temporal Edge Detection Filters in Human Vision
Pontus Ebelin, Gyorgy Denes, Tomas Akenine-Möller, Kalle Åström, Magnus Oskarsson, William H. McIlhagga
ACM Transactions on Applied Perception

2023

HyPoradise: An Open Baseline for Generative Speech Recognition with Large Language Models
Chen Chen, YuChen Hu, Huck Yang, Sabato Marco Siniscalchi, Pin-Yu Chen, Ensiong Chng
NeurIPS 2023
Whispering LLaMA: A Cross-Modal Generative Error Correction Framework for Speech Recognition
Srijith Radhakrishnan, Huck Yang, Sumeer Khan, Rohit Kumar, Narsis Kiani, David Gomez-Cabrero, Jesper Tegnér
EMNLP
Constant Field of View Display Size Effects on First-Person Aiming Time
Josef Spjut, Ben Boudaoud, Joohwan Kim
Frontiers in Optics 2023
BundleSDF: Neural 6-DoF Tracking and 3D Reconstruction of Unknown Objects
Bowen Wen, Jonathan Tremblay, Valts Blukis, Stephen Tyree, Thomas Müller, Alex Evans, Dieter Fox, Jan Kautz, Stan Birchfield
CVPR 2023
Subpixel Deblurring of Anti-Aliased Raster Clip Art
Jinfan Yang, Nicholas Vining, Shakiba Kheradmand, Nathan Carr, Leonid Sigal, Alla Sheffer
Computer Graphics Forum (Proc. Eurographics 2023)
Luminance-Preserving and Temporally Stable Daltonization
Pontus Ebelin, Cyril Crassin, Gyorgy Denes, Magnus Oskarsson, Kalle Åström, Tomas Akenine-Möller
Eurographics 2023 Short Papers
Efficient Dataflow Modeling of Peripheral Encoding in the Human Visual System
Rachel Brown, Vasha DuTell, Bruce Walter, Ruth Rosenholtz, Peter Shirley, Morgan McGuire, David Luebke
ACM Transactions on Applied Perception

2022

As-Locally-Uniform-as-Possible Reshaping of Vector Clip Art
Chrystiano Araujo, Nicholas Vining, Enrique Rosales, Giorgio Gori, Alla Sheffer
ACM Transactions on Graphics (SIGGRAPH 2022)
Detecting Viewer-Perceived Intended Vector Sketch Connectivity
Jerry Yin, Chenxi Liu, Rebecca Liu, Nicholas Vining, Helge Rhodin, Alla Sheffer
ACM Transactions on Graphics (SIGGRAPH 2022)
Image Features Influence Reaction Time: A Learned Probabilistic Perceptual Model for Saccade Latency
Budmonde Duinkharjav, Praneeth Chakravarthula, Rachel Brown, Anjul Patney, Qi Sun
ACM Transactions on Graphics (Proceedings of SIGGRAPH 2022)
Best Technical Paper, SIGGRAPH 2022
PredictionNet: Real-Time Joint Probabilistic Traffic Prediction for Planning, Control, and Simulation
Alexey Kamenev, Lirui Wang, Ollin Boer Bohan, Ishwar Kulkarni, Bilal Kartal, Artem Molchanov, Stan Birchfield, David Nister, Nikolai Smolyanskiy
ICRA 2022

2021

StrokeStrip: Joint Parameterization and Fitting of Stroke Clusters
Dave Pagurek van Mossel, Chenxi Liu, Nicholas Vining, Mikhail Bessmeltsev, Alla Sheffer
ACM Transactions on Graphics (Proc. SIGGRAPH 2021)
Known unknowns: Learning novel concepts using exploratory reasoning-by-elimination
Harsh Agrawal, Eli Meirom, Yuval Atzmon, Shie Mannor, Gal Chechik
Proceedings of the Thirty-Seventh Conference on Uncertainty in Artificial Inte…
Oral
Robust Vision-Based Cheat Detection in Competitive Gaming
Aditya Jonnalagadda, Iuri Frosio, Seth Schenider, Morgan McGuire, Joohwan Kim
I3D ’21

2020

MVLidarNet: Real-Time Multi-Class Scene Understanding for Autonomous Driving Using Multiple Views
Ke Chen, Ryan Oldja, Nikolai Smolyanskiy, Stan Birchfield, Alexander Popov, David Wehr, Ibrahim Eden, Joachim Pehserl
IROS 2020
Eccentricity Effects on Blur and Depth Perception
Qi Sun, Fu-Chung Huang, Li-Yi Wei, David Luebke, Arie Kaufman, Joohwan Kim
Optics Express

2019

FirstPersonScience: Quantifying Psychophysics for First Person Shooter Tasks
Josef Spjut, Ben Boudaoud, Kamran Binaee, Zander Majercik, Morgan McGuire, Joohwan Kim
UCI Esports Conference
Reading Speed Decreases for Fast Readers Under Gaze-Contingent Rendering
Rachel Albert, Angelica Godinez, David Luebke
ACM Symposium on Applied Perception 2019 Article No. 21

2018

Towards Virtual Reality Infinite Walking: Dynamic Saccadic Redirection
Qi Sun, Anjul Patney, Li-Yi Wei, Omer Shapira, Jingwan Lu, Paul Asente, Suwen Zhu, Morgan McGuire, David Luebke, Arie Kaufman
ACM Transactions on Graphics (Proceedings of SIGGRAPH) 2018
On the Importance of Stereo for Accurate Depth Estimation: An Efficient Semi-Supervised Deep Neural Network Approach
Nikolai Smolyanskiy, Alexey Kamenev, Stan Birchfield
CVPR 2018 Workshop on Autonomous Driving

2017

Latency Requirements for Foveated Rendering in Virtual Reality
Rachel Albert, Anjul Patney, David Luebke, Joohwan Kim
ACM Transactions on Applied Perception (TAP) - Special Issue SAP 2017

2016

Towards Foveated Rendering for Gaze-Tracked Virtual Reality
Anjul Patney, Marco Salvi, Joohwan Kim, Anton Kaplanyan, Chris Wyman, Nir Benty, David Luebke, Aaron Lefohn
ACM Transactions on Graphics (Proceedings of SIGGRAPH Asia)

Researchers

Amrita Mazumdar
Daniel Leibovici
Jae-Hyun Jung
Joohwan Kim
Josef Spjut
Min-Hung Chen
Ming-Yu Liu
Ruth Rosenholtz
Szu-Wei Fu
Tomas Geffner
Valts Blukis
Xinshuo Weng