Loss Functions for Image Restoration with Neural Networks

Neural networks are becoming central in several areas of computer vision and image processing and different architectures have been proposed to solve specific problems. The impact of the loss layer of neural networks, however, has not received much attention in the context of image processing: the default and virtually only choice is L2. In this paper, we bring attention to alternative choices for image restoration. In particular, we show the importance of perceptually-motivated losses when the resulting image is to be evaluated by a human observer.

Interactive Reconstruction of Monte Carlo Image Sequences using a Recurrent Denoising Autoencoder

We describe a machine learning technique for reconstructing image sequences rendered using Monte Carlo methods. Our primary focus is on reconstruction of global illumination with extremely low sampling budgets at interactive rates. Motivated by recent advances in image restoration with deep convolutional networks, we propose a variant of these networks better suited to the class of noise present in Monte Carlo rendering. We allow for much larger pixel neighborhoods to be taken into account, while also improving execution speed by an order of magnitude.

Fusing State Spaces for Markov Chain Monte Carlo Rendering

Rendering algorithms using Markov chain Monte Carlo (MCMC) currently build upon two different state spaces. One of them is the path space, where the algorithms operate on the vertices of actual transport paths. The other state space is the primary sample space, where the algorithms operate on sequences of numbers used for generating transport paths. While the two state spaces are related by the sampling procedure of transport paths, all existing MCMC rendering algorithms are designed to work within only one of the state spaces.

Parallel Depth-First Search for Directed Acyclic Graphs

Depth-First Search (DFS) is a pervasive algorithm, often used as a building block for topological sort, connectivity and planarity testing, among many other applications. We propose a novel work-efficient parallel algorithm for the DFS traversal of directed acyclic graph (DAG). The algorithm traverses the entire DAG in a BFS-like fashion no more than three times. As a result it finds the DFS pre-order (discovery) and post-order (finish time) as well as the parent relationship associated with every node in a DAG. We analyse the runtime and work complexity of this novel parallel algorithm.

Polarimetric Multi-view Stereo

Multi-view stereo relies on feature correspondences for
3D reconstruction, and thus is fundamentally flawed in dealing
with featureless scenes. In this paper, we propose polarimetric
multi-view stereo, which combines per-pixel photometric
information from polarization with epipolar constraints
from multiple views for 3D reconstruction. Polarization
reveals surface normal information, and is thus helpful
to propagate depth to featureless regions.

Dynamic Facial Analysis: From Bayesian Filtering to Recurrent Neural Networks

Facial analysis in videos, including head pose estimation and facial landmark localization, is key for many applications such as facial animation capture, human activity recognition, and human-computer interaction. In this paper, we propose to use a recurrent neural network (RNN) for joint estimation and tracking of facial features in videos. We are inspired by the fact that the computation performed in an RNN bears resemblance to Bayesian filters, which have been used for tracking in many previous methods for facial analysis from videos.

Jacob Munkberg

Jacob Munkberg joined NVIDIA Research in 2016 and is part of the real-time rendering research group. Prior to joining NVIDIA, Jacob worked as senior research scientist in Intel’s Advanced Rendering Technology team. He joined Intel in 2008 via Intel's acquisition of the computer graphics startup company, Swiftfoot Graphics, specializing in culling technology and efficient multi-view graphics. Jacob received his Ph.D. in computer science from Lund University and his M.S. in engineering physics from Chalmers University of Technology.

Jacob is currently working on machine learning for real-time graphics applications.

A complete list of prior publications can be found here.



Subscribe to Research RSS