1. [Publications](/index.php/publications)
2. Coherent 3D Portrait Video Reconstruction via Triplane Fusion
 
 # Coherent 3D Portrait Video Reconstruction via Triplane Fusion

  ![](/sites/default/files/styles/wide/public/publications/Teaser_1.jpg?itok=Jr7oZcnz)

 Recent breakthroughs in single-image 3D portrait reconstruction have enabled telepresence systems to stream 3D portrait videos from a single camera in real-time, democratizing telepresence. However, per-frame 3D reconstruction exhibits temporal inconsistency and forgets the user’s appearance. On the other hand, self-reenactment methods can render coherent 3D portraits by driving a 3D avatar built from a single reference image but fail to faithfully preserve the user’s per-frame appearance (e.g., instantaneous facial expressions and lighting). As a result, neither of these two frameworks is an ideal solution for democratized 3D telepresence. In this work, we address this dilemma and propose a novel solution that maintains both coherent identity and dynamic per-frame appearance to enable the best possible realism. To this end, we propose a new fusionbased method that takes the best of both worlds by fusing a canonical 3D prior from a reference view with dynamic appearance from per-frame input views, producing temporally stable 3D videos with faithful reconstruction of the user’s per-frame appearance. Trained only using synthetic data produced by an expression-conditioned 3D GAN, our encoder-based method achieves both state-of-the-art 3D reconstruction and temporal consistency on in-studio and inthe-wild datasets.



 ## Authors



Shengze Wang (NVIDIA)

[Xueting Li](/index.php/person/xueting-li)

[Chao Liu](/index.php/person/chao-liu)

Matthew Chan (NVIDIA)

[Michael Stengel](/index.php/person/michael-stengel)

Henry Fuchs (UNC Chapel Hill)

[Shalini De Mello](/index.php/person/shalini-de-mello)

[Koki Nagano](/index.php/person/koki-nagano)

 

 

 ## Publication Date



Friday, June 13, 2025

 

 ## Published in



[IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2025](https://openaccess.thecvf.com/content/CVPR2025/papers/Wang_Coherent_3D_Portrait_Video_Reconstruction_via_Triplane_Fusion_CVPR_2025_paper.pdf)

 

 ## Research Area



[Artificial Intelligence and Machine Learning ](/index.php/research-area/machine-learning-artificial-intelligence)

[Computer Graphics](/index.php/research-area/computer-graphics)

[Computer Vision](/index.php/research-area/computer-vision)

[Generative AI](/index.php/research-area/generative-ai)

[VR, AR and Display Technology](/index.php/research-area/virtual-augmented-reality)

 

 

 ## External Links



[Project Page](https://research.nvidia.com/labs/amri/projects/coherent3d/)

[ArXiv](https://arxiv.org/abs/2412.08684)

 

 

 ## Uploaded Files



[Paper](https://d1qx31qr3h6wln.cloudfront.net/publications/Wang_Coherent_3D_Portrait_Video_Reconstruction_via_Triplane_Fusion_CVPR_2025_paper.pdf "Open file in new window")7.75 MB