Research Areas

Human Computer Interaction

Associated Publications

2025

Modeling Visually-Guided Aim-and-Shoot behavior in First-Person Shoters
June-Seop Yoon, Hee-Seung Moon, Ben Boudaoud, Josef Spjut, Iuri Frosio, Byungjoo Lee, Joohwan Kim
International Journal of Human-Computer Studies

2024

SpecTrack: Learned Multi-Rotation Tracking via Speckle Imaging
Ziyang Chen, Doğa Doğan, Josef Spjut, Kaan Akşit
SIGGRAPH Asia Posters
Honorable Mention
Fast Explicit-Input Assistance for Teleoperation in Clutter
Nick Walker, Xuning Yang, Animesh Garg, Maya Cakmak, Dieter Fox, Claudia Pérez D’Arpino
2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)
Learning to Move Like Professional Counter-Strike Players
David Durst, F. Xie, V. Sarukkai, Brennan Shacklett, Iuri Frosio, Chen Tessler, Joohwan Kim, C. Taylor, G. Bernstein, S. Choudhury, P. Hanrahan,, Kayvon Fatahalian
The 23rd ACM SIGGRAPH / Eurographics Symposium on Computer Animation (SCA 2024)
Variable Frame Timing Affects Perception of Smoothness in First-Person Gaming
Devi Klein, Josef Spjut, Ben Boudaoud, Joohwan Kim
IEEE Conference on Games (CoG)
Studying Esports Competition: Piloting Methodology for User Studies During Tournaments
Arjun Madhusudan, Josef Spjut, Ben Boudaoud, Joohwan Kim, Benjamin Watson
ACM SIGGRAPH Talks
On Smoothly Varying Frame Timing in First-Person Gaming
UC Santa Barbara and NVIDIA, Josef Spjut, Ben Boudaoud, Joohwan Kim
ACM SIGGRAPH Talks
The Effects of Network Latency on the Peeker's Advantage in First-person Shooter Games
Samin Shahriar Tokey, Zesheng Chen, Colin Mettler, Dexuan Tang, Ben Boudaoud, Joohwan Kim, Josef Spjut, Mark Claypool
Foundations of Digital Games
Is Less More? Rendering for Esports
Benjamin Watson, Josef Spjut, Joohwan Kim, Byungjoo Lee, Mijin Yoo, Peter Shirley, Rulon Raymond
IEEE Computer Graphics and Applications
Interactive Texture Painting with Generative AI
Anita Hu, Nishkrit Desai, Ashley Goldstein, Hassan Abu Alhaija, Seung Wook Kim, Daniela Hasenbring, Alex Zook, Rajeev Rao, Maria Shugrina
SIGGRAPH Asia 2023

2023

Rethinking Display Requirements for Esports and High Interactivity Applications
Joohwan Kim, Josef Spjut, Ben Boudaoud, Benjamin Watson, Turner Whitted
SID
Less is More: Rendering for Esports
Benjamin Watson, Joohwan Kim, Josef Spjut, Peter Shirley, Byungjoo Lee
SIGGRAPH Frontiers Workshop
Interactive AI Material Generation and Editing in NVIDIA Omniverse
Hassan Abu Alhaija, Alex Zook, Michael Babcock, James Lucas, David Tyner, Rajeev Rao, Maria Shugrina
SIGGRAPH 2023
Real-Time Live! Best in Show
Mouse Sensitivity in First-person Targeting Tasks
Ben Boudaoud, Josef Spjut, Joohwan Kim
IEEE Transactions on Games
Affordance Diffusion: Synthesizing Hand-Object Interactions
Yufei Ye, Xueting Li, Abhinav Gupta, Shalini De Mello, Stan Birchfield, Jiaming Song, Shubham Tulsiani, Sifei Liu
IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2023
GazeNeRF: 3D-Aware Gaze Redirection with Neural Radiance Fields
Alessandro Ruzzi, Xiangwei Shi, Xi Wang, Gengyan Li, Shalini De Mello, Hyung Jin Chang, Xucong Zhang, Otmar Hilliges
IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2023
Zero-shot Pose Transfer for Unrigged Stylized 3D Characters
Jiashun Wang, Xueting Li, Sifei Liu, Shalini De Mello, Orazio Gallo, Xiaolong Wang, Jan Kautz
IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2023

2022

Display Size and Targeting Performance: Small Hurts, Large May Help
Joohwan Kim, Arjun Madhusudan, Benjamin Watson, Ben Boudaoud, Roland Tarrazo, Josef Spjut
SIGGRAPH Asia 2022
Esports and expertise: what competitive gaming can teach us about mastery
Ben Boudaoud, Josef Spjut, Joohwan Kim, Arjun Madhusudan, Benjamin Watson
ACM Interactions
Mouse Sensitivity in First-person Targeting Tasks
Ben Boudaoud, Josef Spjut, Joohwan Kim
IEEE Conference on Games
The Esports Frontier: Rendering, Interaction and Display
Benjamin Watson, Josef Spjut, Byungjoo Lee, Greg Ward
SIGGRAPH Frontiers Workshop
Image Features Influence Reaction Time: A Learned Probabilistic Perceptual Model for Saccade Latency
Budmonde Duinkharjav, Praneeth Chakravarthula, Rachel Brown, Anjul Patney, Qi Sun
ACM Transactions on Graphics (Proceedings of SIGGRAPH 2022)
Best Technical Paper, SIGGRAPH 2022
FirstPersonScience: An Open Source Tool for Studying FPS Esports Aiming
Ben Boudaoud, Josef Spjut, Pyarelal Knowles, Arjun Madhusudan, Joohwan Kim
SIGGRAPH Talks
Esports meets human-computer interaction
Simone Kriglstein, Anna Lisa Martin-Niedecken, Josef Spjut, Nicole B. Damen, Selen Turkay, Anders Drachen
ACM Interactions
Model Predictive Control for Fluid Human-to-Robot Handovers
Wei Yang, Balakumar Sundaralingam, Chris Paxton, Iretiayo Akinola, Yu-Wei Chao, Maya Cakmak, Dieter Fox
IEEE International Conference on Robotics and Automation (ICRA) 2022

2021

AdaptiBrush: Adaptive General and Predictable VR Ribbon Brush
Enrique Rosales, Chrystiano Araujo, Jafet Rodriguez, Nicholas Vining, Dongwook Yoon, Alla Sheffer
ACM Transactions on Graphics (SIGGRAPH Asia 2021)
Gaming at Warp Speed: Improving Aiming with Late Warp
Ben Boudaoud, Pyarelal Knowles, Joohwan Kim, Josef Spjut
SIGGRAPH Emerging Technologies
StrokeStrip: Joint Parameterization and Fitting of Stroke Clusters
Dave Pagurek van Mossel, Chenxi Liu, Nicholas Vining, Mikhail Bessmeltsev, Alla Sheffer
ACM Transactions on Graphics (Proc. SIGGRAPH 2021)
Weakly-Supervised Physically Unconstrained Gaze Estimation
Rakshit Kothari, Shalini De Mello, Umar Iqbal, Wonmin Byeon, Seonwook Park, Jan Kautz
IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2021
Oral
A Case Study of First Person Aiming at Low Latency for Esports
Josef Spjut, Ben Boudaoud, Joohwan Kim
Esports and High Performance HCI Workshop
Robust Vision-Based Cheat Detection in Competitive Gaming
Aditya Jonnalagadda, Iuri Frosio, Seth Schenider, Morgan McGuire, Joohwan Kim
I3D ’21

2020

Weakly-Supervised 3D Hand Pose Estimation via Biomechanical Constraints
Adrian Spurr, Umar Iqbal, Pavlo Molchanov, Otmar Hilliges, Jan Kautz
European Conference on Computer Vision, 2020
Post-Render Warp with Late Input Sampling Improves Aiming Under High Latency Conditions
Joohwan Kim, Pyarelal Knowles, Josef Spjut, Ben Boudaoud, Morgan McGuire
High Performance Graphics 2020
Eccentricity Effects on Blur and Depth Perception
Qi Sun, Fu-Chung Huang, Li-Yi Wei, David Luebke, Arie Kaufman, Joohwan Kim
Optics Express
Toward Standardized Classification of Foveated Displays
Josef Spjut, Ben Boudaoud, Jonghyun Kim, Trey Greer, Rachel Albert, Michael Stengel, Kaan Akşit, David Luebke
IEEE Transactions on Visualization and Computer Graphics “Presented at IEEE VR …

2019

Joint Optimization for Cooperative Image Captioning
Gilad Vered, Gal Oren, Yuval Atzmon, Gal Chechik
International conference on computer vision (ICCV)
Latency of 30 ms Benefits First Person Targeting Tasks More Than Refresh Rate Above 60 Hz
Josef Spjut, Ben Boudaoud, Kamran Binaee, Jonghyun Kim, Zander Majercik, Morgan McGuire, David Luebke, Joohwan Kim
Siggraph Asia Technical Briefs
Content-Consistent Generation of Realistic Eyes with Style
Marcel Bühler , Seonwook Park, Shalini De Mello, Xucong, Otmar Hilliges
International Conference on Computer Vision Workshop (ICCVW) 2019
Winner (1st place) Synthetic Eye Generation Challenge
Few-Shot Adaptive Gaze Estimation
Seonwook Park, Shalini De Mello, Pavlo Molchanov, Umar Iqbal, Otmar Hilliges, Jan Kautz
International Conference on Computer Vision (ICCV) 2019
Oral
FirstPersonScience: Quantifying Psychophysics for First Person Shooter Tasks
Josef Spjut, Ben Boudaoud, Kamran Binaee, Zander Majercik, Morgan McGuire, Joohwan Kim
UCI Esports Conference
Reading Speed Decreases for Fast Readers Under Gaze-Contingent Rendering
Rachel Albert, Angelica Godinez, David Luebke
ACM Symposium on Applied Perception 2019 Article No. 21
Esports Arms Race: Latency and Refresh Rate for Competitive Gaming Tasks
Joohwan Kim, Josef Spjut, Morgan McGuire, Zander Majercik, Ben Boudaoud, Rachel Albert, David Luebke
Journal of Vision
Matching Prescription & Visual Acuity: Towards AR for Humans
Jonghyun Kim, Michael Stengel, Jui-Yi Wu, Ben Boudaoud, Josef Spjut, Kaan Akşit, Rachel Albert, Youngmo Jeong, Trey Greer, Ward Lopes, Zander Majercik, Peter Shirley, Morgan McGuire, David Luebke
SIGGRAPH 2019 Emerging Technology
Best in Show Award - Emerging Technology in SIGGRAPH 2019
SIDOD: A Synthetic Image Dataset for 3D Object Pose Recognition with Distractors
Mona Jalal, Josef Spjut, Ben Boudaoud, Margrit Betke
WiCV
Near-Eye Display and Tracking Technologies for Virtual and Augmented Reality
George Alex Koulieris, Kaan Akşit, Michael Stengel, Rafał K. Mantiuk, Katerina Mania
Eurographics 2019
NVGaze: An Anatomically-Informed Dataset for Low-Latency, Near-Eye Gaze Estimation
Joohwan Kim, Michael Stengel, Alexander Majercik, Shalini De Mello, David Dunn, Samuli Laine, Morgan McGuire, David Luebke
ACM Conference on Human-Computer-Interaction (CHI) 2019
Informative Object Annotations: Tell Me Something I Don't Know
Lior Bracha, Gal Chechik
Computer Vision and Pattern Recognition

2018

Fluidic Elastomer Actuators for Haptic Interactions in Virtual Reality
Jose Barreiros, Houston Claure, Bryan Peele, Omer Shapira, Josef Spjut, David Luebke, Malte Jung, Robert Shepherd
IEEE Robotics and Automation Letters
FocusAR: Auto-focus Augmented Reality Eyeglasses for both Real World and Virtual Imagery
Praneeth Chakravarthula, David Dunn, Kaan Akşit, Henry Fuchs
IEEE TVCG
Towards Virtual Reality Infinite Walking: Dynamic Saccadic Redirection
Qi Sun, Anjul Patney, Li-Yi Wei, Omer Shapira, Jingwan Lu, Paul Asente, Suwen Zhu, Morgan McGuire, David Luebke, Arie Kaufman
ACM Transactions on Graphics (Proceedings of SIGGRAPH) 2018
Light-weight Head Pose Invariant Gaze Tracking
Rajeev Ranjan, Shalini De Mello, Jan Kautz
IEEE Computer Vision and Pattern Recognition Workshop (CVPRW) 2018
Best Paper (runner up) Workshop on Analysis and Modeling of Faces and Gestures
Depth-Based 3D Hand Pose Estimation: From Current Achievements to Future Goals
Shanxin Yuan, Guillermo Garcia-Hernando, Bjorn Stenger, Pavlo Molchanov, Jan Kautz, Sina Honari
Conference on Computer Vision and Pattern Recognition
A Variable Shape and Variable Stiffness Controller for Haptic Virtual Interactions
Benjamin C. Mac Murray, Bryan N. Peele, Patricia Xu, Josef Spjut, Omer Shapira, David Luebke, Robert F. Shepherd
RoboSoft

2017

Stretchable Transducers for Kinesthetic Interactions in Virtual Reality
Robert Shepher, Bryan Peele, Benjamin Mac Murray, Jose Barreiros, Omer Shapira, Josef Spjut, David Luebke
SIGGRAPH Emerging Technologies

2016

Towards Foveated Rendering for Gaze-Tracked Virtual Reality
Anjul Patney, Marco Salvi, Joohwan Kim, Anton Kaplanyan, Chris Wyman, Nir Benty, David Luebke, Aaron Lefohn
ACM Transactions on Graphics (Proceedings of SIGGRAPH Asia)
Towards Selecting Robust Hand Gestures for Automotive Interfaces
Shalini Gupta, Pavlo Molchanov, Xiaodong Yang, Kihwan Kim, Stephen Tyree, Jan Kautz
IEEE Intelligent Vehicles Symposium

Researchers

Ben Boudaoud
Jae-Hyun Jung
Jaesung Choe
Jean Kossaifi
Josef Spjut
Michael Stengel
Omer Shapira
Peter Xenopoulos
Ruth Rosenholtz
Seonwook Park
Shalini De Mello
Valts Blukis