1. [Publications](/index.php/publications)
2. 3D Reconstruction with Generalizable Neural Fields using Scene Priors
 
 # 3D Reconstruction with Generalizable Neural Fields using Scene Priors

  ![](/sites/default/files/publications/output.gif) 

 High-fidelity 3D scene reconstruction has been substantially advanced by recent progress in neural fields. However, most existing methods train a separate network from scratch for each individual scene. This is not scalable, inefficient, and unable to yield good results given limited views. While learning-based multi-view stereo methods alleviate this issue to some extent, their multi-view setting makes it less flexible to scale up and to broad applications. Instead, we introduce training generalizable Neural Fields incorporating scene Priors (NFPs). The NFP network maps any single-view RGB-D image into signed distance and radiance values. A complete scene can be reconstructed by merging individual frames in the volumetric space WITHOUT a fusion module, which provides better flexibility. The scene priors can be trained on large-scale datasets, allowing for fast adaptation to the reconstruction of a new scene with fewer views. NFP not only demonstrates SOTA scene reconstruction performance and efficiency, but it also supports single-image novel-view synthesis, which is under-explored in neural fields. More qualitative results are available at: <https://oasisyang.github.io/neural-prior>.



 ## Authors



Yang Fu (University of California at San Diego)

[Shalini De Mello](/index.php/person/shalini-de-mello)

[Xueting Li](/index.php/person/xueting-li)

Amey Kulkarni (NVIDIA)

[Jan Kautz](/index.php/person/jan-kautz)

Xiaolong Wang (University of California at San Diego)

[Sifei Liu](/index.php/person/sifei-liu)

 

 

 ## Publication Date



Monday, May 6, 2024

 

 ## Published in



[International Conference on Learning Representations (ICLR) 2024](https://proceedings.iclr.cc/paper_files/paper/2024/hash/0bd32794b26cfc99214b89313764da8e-Abstract-Conference.html)

 

 ## Research Area



[Artificial Intelligence and Machine Learning ](/index.php/research-area/machine-learning-artificial-intelligence)

[Computer Vision](/index.php/research-area/computer-vision)

[Generative AI](/index.php/research-area/generative-ai)

 

 

 ## External Links



[Project Page](https://oasisyang.github.io/neural-prior/)

[ArXiv](https://arxiv.org/abs/2309.15164v2)

[Video](https://youtu.be/cqVzTk3U6e4)

 

 

 ## Uploaded Files



[Paper](https://d1qx31qr3h6wln.cloudfront.net/publications/ICLR-2024-3d-reconstruction-with-generalizable-neural-fields-using-scene-priors-Paper-Conference.pdf "Open file in new window")27.68 MB

[Supplementary](https://d1qx31qr3h6wln.cloudfront.net/publications/ICLR_2024_Conference_supp.pdf "Open file in new window")6.08 MB