1. [Publications](/publications)
2. NRMVS: Non-Rigid Multi-view Stereo
 
 # NRMVS: Non-Rigid Multi-view Stereo

  ![](/sites/default/files/styles/wide/public/publications/nrmvs.jpg?itok=Vtc7GnaB)

 Multi-view Stereo (MVS) is a common solution in photogrammetry applications for the dense reconstruction of a static scene from images. The static scene assumption, however, limits the general applicability of MVS algorithms, as many day-to-day scenes undergo non-rigid motion, e.g., clothes, faces, or human bodies. In this paper, we open up a new challenging direction: Dense 3D reconstruction of scenes with non-rigid changes observed from a small number of images sparsely captured from different views with a single monocular camera, which we call non-rigid multi-view stereo (NRMVS) problem. We formulate this problem as a joint optimization of deformation and depth estimation, using deformation graphs as the underlying representation. We propose a new sparse 3D to 2D matching technique with a dense patch-match evaluation scheme to estimate the most plausible deformation field satisfying depth and photometric consistency. We show that a dense reconstruction of a scene with non-rigid changes from a few images is possible, and demonstrate that our method can be used to interpolate novel deformed scenes from various combinations of deformation estimates derived from the sparse views.



 ## Authors



Matthias Innmann (University of Erlangen-Nuremberg)

Kihwan Kim (NVIDIA)

Jinwei Gu (SenseTime)

Matthias Niessner (Technical University of Munich)

[Charles Loop](/person/charles-loop)

Marc Stamminger (Friedrich-Alexander University of Erlangen-Nuremberg)

[Jan Kautz](/person/jan-kautz)

 

 

 ## Publication Date



Monday, March 2, 2020

 

 ## Published in



[IEEE Winter Conference on Applications of Computer Vision (WACV ’20)](https://wacv20.wacv.net/)

 

 ## Research Area



[Computer Vision](/research-area/computer-vision)

[VR, AR and Display Technology](/research-area/virtual-augmented-reality)

 

 

 ## External Links



[PDF (proceeding)](http://openaccess.thecvf.com/content_WACV_2020/papers/Innmann_NRMVS_Non-Rigid_Multi-view_Stereo_WACV_2020_paper.pdf)

[Arxiv (PDF)](https://arxiv.org/pdf/1901.03910.pdf)

[Video (youtube)](https://www.youtube.com/watch?v=B4YBWFuYBdE)

 

 

 ## Copyright



This material is posted here with permission of the IEEE. Internal or personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution must be obtained from the IEEE by writing to <pubs-permissions@ieee.org>.