1. [Publications](/index.php/publications) 2. Neural Inverse Rendering of an Indoor Scene from a Single Image # Neural Inverse Rendering of an Indoor Scene from a Single Image ![](/sites/default/files/styles/wide/public/publications/IRN_0_0.jpg?itok=1fSv9p6O) Inverse rendering aims to estimate physical attributes of a scene, e.g., reflectance, geometry, and lighting, from image(s). Inverse rendering has been studied primarily for single objects or with methods that solve for only one of the scene attributes. We propose the first learning based approach that jointly estimates albedo, normals, and lighting of an indoor scene from a single image. Our key contribution is the Residual Appearance Renderer (RAR), which can be trained to synthesize complex appearance effects (e.g., inter-reflection, cast shadows, near-field illumination, and realistic shading), which would be neglected otherwise. This enables us to perform self-supervised learning on real data using a reconstruction loss, based on re-synthesizing the input image from the estimated components. We finetune with real data after pretraining with synthetic data. To this end we use physically-based rendering to synthesize a large-scale training dataset. Experimental results show that our approach outperforms state-of-the-art methods that estimate one or more scene attributes. ## Authors Soumyadip Sengupta (University of Maryland, College Park) Jinwei Gu (SenseTime) Kihwan Kim (NVIDIA) Guilin Liu (NVIDIA) David W. Jacobs (University of Maryland, College Park) [Jan Kautz](/index.php/person/jan-kautz) ## Publication Date Tuesday, October 29, 2019 ## Published in [IEEE International Conference on Computer Vision (ICCV 2019)](http://iccv2019.thecvf.com/) ## Research Area [Computer Vision](/index.php/research-area/computer-vision) [Artificial Intelligence and Machine Learning ](/index.php/research-area/machine-learning-artificial-intelligence) ## External Links [ArXiv (pdf) (HQ)](https://arxiv.org/pdf/1901.02453.pdf) [Soumyadip's homepage](https://homes.cs.washington.edu/~soumya91/) [A project page with raw images of experiments and extra details](https://senguptaumd.github.io/Neural-Inverse-Rendering/) ## Copyright This material is posted here with permission of the IEEE. Internal or personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution must be obtained from the IEEE by writing to .