1. [Publications](/index.php/publications)
2. Two-shot Spatially-varying BRDF and Shape Estimation
 
 # Two-shot Spatially-varying BRDF and Shape Estimation

  ![](/sites/default/files/styles/wide/public/publications/twoshotbrdf.png?itok=qcLEy-8f)

 Capturing the shape and spatially-varying appearance (SVBRDF) of an object from images is a challenging task that has applications in both computer vision and graphics. Traditional optimization-based approaches often need a large number of images taken from multiple views in a controlled environment. Newer deep learning-based approaches require only a few input images, but the reconstruction quality is not on par with optimization techniques. We propose a novel deep learning architecture with a stage-wise estimation of shape and SVBRDF. The previous predictions guide each estimation, and a joint refinement network later refines both SVBRDF and shape. We follow a practical mobile image capture setting and use unaligned two-shot flash and no-flash images as input. Both our two-shot image capture and network inference can run on mobile hardware. We also create a large-scale synthetic training dataset with domain-randomized geometry and realistic materials. Extensive experiments on both synthetic and real-world datasets show that our network trained on a synthetic dataset can generalize well to real-world images. Comparisons with recent approaches demonstrate the superior performance of the proposed approach.



 ## Authors



Mark Boss ( University of Tübingen)

Varun Jampani (NVIDIA)

Kihwan Kim (NVIDIA)

Hendrik P.A. Lensch ( University of Tübingen)

[Jan Kautz](/index.php/person/jan-kautz)

 

 

 ## Publication Date



Sunday, June 14, 2020

 

 ## Published in



[IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2020](http://cvpr2020.thecvf.com/)

 

 ## Research Area



[Computer Vision](/index.php/research-area/computer-vision)

[Artificial Intelligence and Machine Learning ](/index.php/research-area/machine-learning-artificial-intelligence)

 

 

 ## External Links



[Paper](https://arxiv.org/pdf/2004.00403)

[Video (youtube)](https://www.youtube.com/watch?v=Bvjoolt2-iY)

 

 

 ## Copyright



This material is posted here with permission of the IEEE. Internal or personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution must be obtained from the IEEE by writing to <pubs-permissions@ieee.org>.