Research Areas

VR, AR and Display Technology

Associated Publications

2024

Full-colour 3D holographic augmented-reality displays with metasurface waveguides
Manu Gopakumar, Gun-Yeal Lee, Suyeon Choi, Brian Chao, Yifan Peng, Jonghyun Kim, Gordon Wetzstein
Nature

2023

Constant Field of View Display Size Effects on First-Person Aiming Time
Josef Spjut, Ben Boudaoud, Joohwan Kim
Frontiers in Optics 2023
AI-Mediated 3D Video Conferencing
Michael Stengel, Koki Nagano, Chao Liu, Matthew Chan, Alex Trevithick, Shalini De Mello, Jonghyun Kim, David Luebke, Amrita Mazumdar, Shengze Wang, Mayoore Jaiswal
ACM SIGGRAPH Emerging Technologies 2023
Live 3D Portrait: Real-Time Radiance Fields for Single-Image Portrait View Synthesis
Alexander Trevithick, Matthew Chan, Michael Stengel, Eric R. Chan, Chao Liu, Zhiding Yu, Sameh Khamis, Manmohan Chandraker, Ravi Ramamoorthi, Koki Nagano
ACM Transactions On Graphics (SIGGRAPH 2023)
GazeNeRF: 3D-Aware Gaze Redirection with Neural Radiance Fields
Alessandro Ruzzi, Xiangwei Shi, Xi Wang, Gengyan Li, Shalini De Mello, Hyung Jin Chang, Xucong Zhang, Otmar Hilliges
IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2023
BundleSDF: Neural 6-DoF Tracking and 3D Reconstruction of Unknown Objects
Bowen Wen, Jonathan Tremblay, Valts Blukis, Stephen Tyree, Thomas Müller, Alex Evans, Dieter Fox, Jan Kautz, Stan Birchfield
CVPR 2023

2022

Display Size and Targeting Performance: Small Hurts, Large May Help
Joohwan Kim, Arjun Madhusudan, Benjamin Watson, Ben Boudaoud, Roland Tarrazo, Josef Spjut
SIGGRAPH Asia 2022
Embodied Scene-aware Human Pose Estimation
Zhengyi Luo, Shun Iwase, Ye Yuan, Kris Kitani
NeurIPS 2022
The Esports Frontier: Rendering, Interaction and Display
Benjamin Watson, Josef Spjut, Byungjoo Lee, Greg Ward
SIGGRAPH Frontiers Workshop
Time-multiplexed Neural Holography: A Flexible Framework for Holographic Near-eye Displays with Fast Heavily-quantized Spatial Light Modulators
Suyeon Choi, Manu Gopakumar, Yifan Peng, Jonghyun Kim, Matthew O'Toole, Gordon Wetzstein
Holographic Glasses for Virtual Reality
Jonghyun Kim, Manu Gopakumar, Suyeon Choi, Yifan Peng, Ward Lopes, Gordon Wetzstein
SIGGRAPH 2022

2021

AdaptiBrush: Adaptive General and Predictable VR Ribbon Brush
Enrique Rosales, Chrystiano Araujo, Jafet Rodriguez, Nicholas Vining, Dongwook Yoon, Alla Sheffer
ACM Transactions on Graphics (SIGGRAPH Asia 2021)
Unfiltered holography: optimizing high diffraction orders without optical filtering for compact holographic displays
Manu Gopakumar, Jonghyun Kim, Suyeon Choi, Yifan Peng, Gordon Wetzstein
Optics Letters
Neural 3D Holography
Suyeon Choi, Manu Gopakumar, Yifan Peng, Jonghyun Kim, Gordon Wetzstein
ACM Transactions on Graphics (SIGGRAPH Asia 2021)
Speckle-free holography with partially coherent light sources and camera-in-the-loop calibration
Yifan Peng, Suyeon Choi, Jonghyun Kim, Gordon Wetzstein
Science Advances
Enabling Next-generation Holographic Displays with Artificial Intelligence
Gordon Wetzstein, Yifan Peng, Suyeon Choi, Jonghyun Kim
OSA Imaging and Applied Optics Congress 2021 (3D, COSI, DH, ISA, pcAOP)
A distributed, decoupled system for losslessly streaming dynamic light probes to thin clients
Michael Stengel, Zander Majercik, Ben Boudaoud, Morgan McGuire
ACM Multimedia Systems Conference
High-quality Holographic Displays Using Double SLMs and Camera-in-the-loop Optimization
Suyeon Choi, Yifan Peng, Jonghyun Kim, Gordon Wetzstein
Proceedings Volume 11765, Optical Architectures for Displays and Sensing in Aug…
Optimizing Image Quality for Holographic Near-eye Displays with Michelson Holography
Suyeon Choi, Jonghyun Kim, Yifan Peng, Gordon Wetzstein
Optica

2020

Weakly-Supervised 3D Hand Pose Estimation via Biomechanical Constraints
Adrian Spurr, Umar Iqbal, Pavlo Molchanov, Otmar Hilliges, Jan Kautz
European Conference on Computer Vision, 2020
Neural Holography
Yifan Peng, Suyeon Choi, Nitish Padmanaban, Jonghyun Kim, Gordon Wetzstein
SIGGRAPH 2020 Emerging Technology
Post-Render Warp with Late Input Sampling Improves Aiming Under High Latency Conditions
Joohwan Kim, Pyarelal Knowles, Josef Spjut, Ben Boudaoud, Morgan McGuire
High Performance Graphics 2020
Prescription AR: a fully-customized prescription-embedded augmented reality display
Jui-Yi Wu, Jonghyun Kim
Optics Express
NRMVS: Non-Rigid Multi-view Stereo
Matthias Innmann, Kihwan Kim, Jinwei Gu, Matthias Niessner , Charles Loop, Marc Stamminger, Jan Kautz
IEEE Winter Conference on Applications of Computer Vision (WACV ’20)
Eccentricity Effects on Blur and Depth Perception
Qi Sun, Fu-Chung Huang, Li-Yi Wei, David Luebke, Arie Kaufman, Joohwan Kim
Optics Express
Toward Standardized Classification of Foveated Displays
Josef Spjut, Ben Boudaoud, Jonghyun Kim, Trey Greer, Rachel Albert, Michael Stengel, Kaan Akşit, David Luebke
IEEE Transactions on Visualization and Computer Graphics “Presented at IEEE VR …
Patch scanning displays: spatiotemporal enhancement for displays
Kaan Akşit
OSA Optics Express

2019

Latency of 30 ms Benefits First Person Targeting Tasks More Than Refresh Rate Above 60 Hz
Josef Spjut, Ben Boudaoud, Kamran Binaee, Jonghyun Kim, Zander Majercik, Morgan McGuire, David Luebke, Joohwan Kim
Siggraph Asia Technical Briefs
Content-Consistent Generation of Realistic Eyes with Style
Marcel Bühler , Seonwook Park, Shalini De Mello, Xucong, Otmar Hilliges
International Conference on Computer Vision Workshop (ICCVW) 2019
Winner (1st place) Synthetic Eye Generation Challenge
Reading Speed Decreases for Fast Readers Under Gaze-Contingent Rendering
Rachel Albert, Angelica Godinez, David Luebke
ACM Symposium on Applied Perception 2019 Article No. 21
Esports Arms Race: Latency and Refresh Rate for Competitive Gaming Tasks
Joohwan Kim, Josef Spjut, Morgan McGuire, Zander Majercik, Ben Boudaoud, Rachel Albert, David Luebke
Journal of Vision
Matching Prescription & Visual Acuity: Towards AR for Humans
Jonghyun Kim, Michael Stengel, Jui-Yi Wu, Ben Boudaoud, Josef Spjut, Kaan Akşit, Rachel Albert, Youngmo Jeong, Trey Greer, Ward Lopes, Zander Majercik, Peter Shirley, Morgan McGuire, David Luebke
SIGGRAPH 2019 Emerging Technology
Best in Show Award - Emerging Technology in SIGGRAPH 2019
Foveated AR: Dynamically-Foveated Augmented Reality Display
Jonghyun Kim, Youngmo Jeong, Michael Stengel, Kaan Akşit, Rachel Albert, Ben Boudaoud, Trey Greer, Joohwan Kim, Ward Lopes, Zander Majercik, Peter Shirley, Josef Spjut, Morgan McGuire, David Luebke
SIGGRAPH 2019 Technical Paper
SIDOD: A Synthetic Image Dataset for 3D Object Pose Recognition with Distractors
Mona Jalal, Josef Spjut, Ben Boudaoud, Margrit Betke
WiCV
Near-Eye Display and Tracking Technologies for Virtual and Augmented Reality
George Alex Koulieris, Kaan Akşit, Michael Stengel, Rafał K. Mantiuk, Katerina Mania
Eurographics 2019
NVGaze: An Anatomically-Informed Dataset for Low-Latency, Near-Eye Gaze Estimation
Joohwan Kim, Michael Stengel, Alexander Majercik, Shalini De Mello, David Dunn, Samuli Laine, Morgan McGuire, David Luebke
ACM Conference on Human-Computer-Interaction (CHI) 2019
Manufacturing Application-Driven Foveated Near-Eye Displays
Kaan Akşit, Praneeth Chakravarthula, Kishore Rathinavel, Youngmo Jeong, Rachel Albert, Henry Fuchs, David Luebke
IEEE transactions on visualization and computer graphics

2018

Fluidic Elastomer Actuators for Haptic Interactions in Virtual Reality
Jose Barreiros, Houston Claure, Bryan Peele, Omer Shapira, Josef Spjut, David Luebke, Malte Jung, Robert Shepherd
IEEE Robotics and Automation Letters
FocusAR: Auto-focus Augmented Reality Eyeglasses for both Real World and Virtual Imagery
Praneeth Chakravarthula, David Dunn, Kaan Akşit, Henry Fuchs
IEEE TVCG
Steerable application-adaptive near eye displays
Kishore Rathinavel, Praneeth Chakravarthula, Kaan Akşit, Josef Spjut, Ben Boudaoud, Turner Whitted, David Luebke, Henry Fuchs
Proceeding SIGGRAPH '18 ACM SIGGRAPH 2018 Emerging Technologies
Towards Virtual Reality Infinite Walking: Dynamic Saccadic Redirection
Qi Sun, Anjul Patney, Li-Yi Wei, Omer Shapira, Jingwan Lu, Paul Asente, Suwen Zhu, Morgan McGuire, David Luebke, Arie Kaufman
ACM Transactions on Graphics (Proceedings of SIGGRAPH) 2018
Super SloMo: High Quality Estimation of Multiple Intermediate Frames for Video Interpolation
Huaizu Jiang, Deqing Sun, Varun Jampani, Ming-Hsuan Yang, Erik Learned-Miller, Jan Kautz
CVPR 2018
A Variable Shape and Variable Stiffness Controller for Haptic Virtual Interactions
Benjamin C. Mac Murray, Bryan N. Peele, Patricia Xu, Josef Spjut, Omer Shapira, David Luebke, Robert F. Shepherd
RoboSoft

2017

Near-Eye Varifocal Augmented Reality Display using See-Through Screens
Kaan Akşit, Ward Lopes, Jonghyun Kim, Peter Shirley, David Luebke
ACM Siggraph Asia 2017
Perceptually-Guided Foveation for Light Field Displays
Qi Sun, Fu-Chung Huang, Joohwan Kim, Li-Yi Wei, David Luebke, Arie Kaufman
ACM SIGGRAPH ASIA 2017
Near-eye Light Field Holographic Rendering with Spherical Waves for Wide Field of View Interactive 3D Computer Graphics
Liang Shi, Fu-Chung Huang, Ward Lopes, Wojciech Matusik, David Luebke
ACM SIGGRAPH ASIA 2017
Intrinsic3D: High-Quality 3D Reconstruction by Joint Appearance and Geometry Optimization with Spatially-Varying Lighting
Robert Maier, Kihwan Kim, Daniel Cremers, Jan Kautz, Matthias Nießner
IEEE International Conference on Computer Vision (ICCV 2017)
Latency Requirements for Foveated Rendering in Virtual Reality
Rachel Albert, Anjul Patney, David Luebke, Joohwan Kim
ACM Transactions on Applied Perception (TAP) - Special Issue SAP 2017
Varifocal Virtuality: A Novel Optical Layout for Near-Eye Display
Kaan Akşit, Ward Lopes, Jonghyun Kim, Josef Spjut, Peter Shirley, David Luebke
ACM Siggraph Emerging Technologies
Membrane AR: Varifocal, Wide-Field-of-View Augmented Reality Display from Deformable Membranes
David Dunn, Kaan Akşit, David Luebke
ACM Siggraph Emerging Technologies
Mixed-primary Factorization for Dual-frame Computational Displays
Fu-Chung Huang, David Pajak, Jonghyun Kim, Jan Kautz, David Luebke
ACM SIGGRAPH
Stretchable Transducers for Kinesthetic Interactions in Virtual Reality
Robert Shepher, Bryan Peele, Benjamin Mac Murray, Jose Barreiros, Omer Shapira, Josef Spjut, David Luebke
SIGGRAPH Emerging Technologies
Light Fields for Near-eye Displays
Fu-Chung Huang
IEEE COMSOC MMTC Communications - Frontiers
Wide Field Of View Varifocal Near-Eye Display Using See-Through Deformable Membrane Mirrors
David Dunn, Kaan Akşit, David Luebke
IEEE Transactions on Visualization and Computer Graphics

2016

Towards Foveated Rendering for Gaze-Tracked Virtual Reality
Anjul Patney, Marco Salvi, Joohwan Kim, Anton Kaplanyan, Chris Wyman, Nir Benty, David Luebke, Aaron Lefohn
ACM Transactions on Graphics (Proceedings of SIGGRAPH Asia)
Hybrid Modulation for Near Zero Display Latency
Trey Greer, Josef Spjut, David Luebke, Turner Whitted

2015

The Light Field Stereoscope
Fu-Chung Huang, David Luebke, Gordon Wetzstein
ACM SIGGRAPH 2015 Emerging Technologies
Filtering Environment Illumination for Interactive Physically-Based Rendering in Mixed Reality
Soham Uday Mehta, Kihwan Kim, Dawid Pajak, Kari Pulli, Jan Kautz, Ravi Ramamoorthi
Eurographics Symposium on Rendering (EGSR 2015)
Slim near eye display using pinhole aperture arrays
Kaan Akşit, Jan Kautz, David Luebke
Applied Optics

2014

DT-SLAM: Deferred Triangulation for Robust SLAM
Daniel Herrera C., Kihwan Kim, Juho Kannala, Kari Pulli, Janne Heikkila¨
IEEE International Conference in 3DV (3D Vision)
Cascaded Displays: Spatiotemporal Superresolution using Offset Pixel Layers
Felix Heide, Douglas Lanman, Dikpal Reddy, Jan Kautz, Kari Pulli, David Luebke
ACM Transactions on Graphics (Proceedings of SIGGRAPH)
FoundationPose: Unified 6D Pose Estimation and Tracking of Novel Objects
Bowen Wen, Wei Yang, Jan Kautz, Stan Birchfield

Researchers

Amrita Mazumdar
Ben Boudaoud
Bowen Wen
Charles Loop
Chen-Hsuan Lin
David Luebke
Erik Lindholm
Jae-Hyun Jung
Jonghyun Kim
Joohwan Kim
Josef Spjut
Marco Salvi
Mark Kilgard
Michael Stengel
Ming-Yu Liu
Omer Shapira
Tomas Akenine-Möller
Ward Lopes