1. [Publications](/index.php/publications)
2. Live 3D Portrait: Real-Time Radiance Fields for Single-Image Portrait View Synthesis
 
 # Live 3D Portrait: Real-Time Radiance Fields for Single-Image Portrait View Synthesis

  ![](/sites/default/files/styles/wide/public/publications/papers_896s2-file2_0.jpg?itok=reEHTD7t)

 We present a one-shot method to infer and render a photorealistic 3D representation from a single unposed image (e.g., face portrait) in real-time. Given a single RGB input, our image encoder directly predicts a canonical triplane representation of a neural radiance field for 3D-aware novel view synthesis via volume rendering. Our method is fast (24 fps) on consumer hardware, and produces higher quality results than strong GAN-inversion baselines that require test-time optimization. To train our triplane encoder pipeline, we use only synthetic data, showing how to distill the knowledge from a pretrained 3D GAN into a feedforward encoder. Technical contributions include a Vision Transformer-based triplane encoder, a camera data augmentation strategy, and a well-designed loss function for synthetic data training. We benchmark against the state-of-the-art methods, demonstrating significant improvements in robustness and image quality in challenging real-world settings. We showcase our results on portraits of faces (FFHQ) and cats (AFHQ), but our algorithm can also be applied in the future to other categories with a 3D-aware image generator.



 ## Authors



Alexander Trevithick (UC San Diego)

Matthew Chan (NVIDIA)

[Michael Stengel](/index.php/person/michael-stengel)

Eric R. Chan (Stanford University)

[Chao Liu](/index.php/person/chao-liu)

[Zhiding Yu](/index.php/person/zhiding-yu)

Sameh Khamis (NVIDIA)

Manmohan Chandraker (UC San Diego)

Ravi Ramamoorthi (UC San Diego)

[Koki Nagano](/index.php/person/koki-nagano)

 

 

 ## Publication Date



Sunday, August 6, 2023

 

 ## Published in



[ACM Transactions On Graphics (SIGGRAPH 2023)](https://s2023.siggraph.org/)

 

 ## Research Area



[Artificial Intelligence and Machine Learning ](/index.php/research-area/machine-learning-artificial-intelligence)

[Computer Graphics](/index.php/research-area/computer-graphics)

[Computer Vision](/index.php/research-area/computer-vision)

[Generative AI](/index.php/research-area/generative-ai)

[VR, AR and Display Technology](/index.php/research-area/virtual-augmented-reality)

 

 

 ## External Links



[Project](https://research.nvidia.com/labs/nxp/lp3d/)

 

 

 ## Uploaded Files



[Video](https://d1qx31qr3h6wln.cloudfront.net/publications/Main_140s.mp4 "Open video in new window")269.17 MB