Light Fields for Near-eye Displays

The most important requirement to make any near-eye display successful is to provide a comfortable visual experience. This requirement has many boxes to check: having high resolution and wide field of view, being lightweight, having small form factor, and supporting focus cue. Like 3D TVs and movies, near-eye displays also need to solve the vergence and accommodation conflicts. In current Virtual Reality (VR) displays, the user fixates his focus on the fixed focal plane, and the disparity in the pre-processed content drives the eye to verge and creates a 3D sensation.

The Light Field Stereoscope

Over the last few years, virtual reality has re-emerged as a technology that is now feasible at low cost via inexpensive cellphone components. In particular, advances of high-resolution micro displays, low-latency orientation trackers, and modern GPUs facilitate extremely immersive experiences. To facilitate comfortable long-term experiences and wide-spread user acceptance, however, the vergence-accommodation conflict inherent to all stereoscopic displays will have to be solved.

Feedforward and Recurrent Neural Networks Backward Propagation and Hessian in Matrix Form

In this paper we focus on the linear algebra theory behind feedforward (FNN) and recurrent (RNN) neural networks. We review backward propagation, including backward propagation through time (BPTT). Also, we obtain a new exact expression for Hessian, which represents second order effects. We show that for t time steps the weight gradient can be expressed as a rank-t matrix, while the weight Hessian is as a sum of t2 Kronecker products of rank-1 and WTAW matrices, for some matrix A and weight matrix W.

Benjamin Klenk

Benjamin Klenk joined NVIDIA research in July 2017 after completing his PhD studies in Computer Engineering at the Ruprecht-Karls University in Heidelberg, Germany (defended with summa cum laude in January 2018). His thesis focused on communication models for direct inter-GPU communication in which the GPU orchestrates the network traffic itself without the intervention of the CPU. He also holds a B.Eng. in Electrial Engineering from the Cooperative State Universtiy Mosbach, Germany and a M.Sc. in Computer Engineering from the Ruprecht-Karls University in Heidelberg, Germany.

Benjamin is a member of the Networking Research Group and his research focuses on GPU-centric networking archtictures, communication models, and distributed and parallel applications with a focus on the training of deep neural nets.

Main Field of Interest: 

Varun Jampani

I am a research scientist at Nvidia Research in Westford, US. Prior to joining Nvidia, I completed my PhD at Perceiving Systems department, Max-Planck Institute (MPI) for Intelligent Systems in Tübingen, Germany. I did my bachelors and masters in Computer Science at IIIT-Hyderabad, India.

My work lies at the intersection of Computer Vision and Machine Learning. Specifically, I am working on leveraging machine learning techniques for better inference in computer vision models. The main research question is how to make use of learning techniques such as deep neural networks and random forests for inference in structured prediction frameworks.

See my personal webpage for more details:

Main Field of Interest: 

Michael Stengel

Michael Stengel joined Nvidia as a Research Scientist in 2017.

His research is focused on perceptual aspects in Computer Graphics, in particular hardware and algorithms for gaze-contingent real-time rendering.

Michael Stengel received a Diploma in Computational Visualistics from University of Magdeburg, Germany (2011) and holds a Ph.D. degree in Computer Science from TU Braunschweig, Germany (2016).

In 2010 he joined the Virtual Reality Lab at Volkswagen AG, Wolfsburg, Germany where he developed haptics algorithms for immersive rendering.

As a postdoctoral research scientist he joined in 2016 TU Delft and VU Medical Center, Amsterdam in the Netherlands where he developed a Virtual Reality headset with gaze tracking for subject monitoring during awake brain surgeries.


Yaosheng Fu

Yaosheng Fu joined NVIDIA in September, 2017 as a member of the architecture research team. His current interests include computer architecture, memory systems and parallel computing. Yaosheng received his Ph.D. degree in Electrical Engineering at Princeton University, NJ in 2017 and B.S. in Electronic Engineering at Tsinghua University, China in 2010.

Near-eye Light Field Holographic Rendering with Spherical Waves for Wide Field of View Interactive 3D Computer Graphics

Holograms have high resolution and great depth of field allowing the eye to view a scene much like seeing through a virtual window. Unfortunately, computer generated holography (CGH) does not deliver the same promise due to hardware limitations under plane wave illumination and large computational cost. Light field displays have been popular due to their capability to provide continuous focus cue. However, light field displays suffer from the trade offs between spatial and angular resolution, and do not model diffraction.

Perceptually-Guided Foveation for Light Field Displays

A variety of applications such as virtual reality and immersive cinema require high image quality, low rendering latency, and consistent depth cues. 4D light field displays support focus accommodation, but are more costly to render than 2D images, resulting in higher latency. The human visual system can resolve higher spatial frequencies in the fovea than in the periphery. This property has been harnessed by recent 2D foveated rendering methods to reduce computation cost while maintaining perceptual quality.


Subscribe to Research RSS