1. [Publications](/publications)
2. One-Shot Free-View Neural Talking-Head Synthesis for Video Conferencing
 
 # One-Shot Free-View Neural Talking-Head Synthesis for Video Conferencing

  ![](/sites/default/files/publications/teaser_1.gif) 

 We propose a neural talking-head video synthesis model and demonstrate its application to video conferencing. Our model learns to synthesize a talking-head video using a source image containing the target person's appearance and a driving video that dictates the motion in the output. Our motion is encoded based on a novel keypoint representation, where the identity-specific and motion-related information is decomposed unsupervisedly. Extensive experimental validation shows that our model outperforms competing methods on benchmark datasets. Moreover, our compact keypoint representation enables a video conferencing system that achieves the same visual quality as the commercial H.264 standard while only using one-tenth of the bandwidth. Besides, we show our keypoint representation allows the user to rotate the head during synthesis, which is useful for simulating a face-to-face video conferencing experience.



 ## Authors



[Ting-Chun Wang](/person/ting-chun-wang)

Arun Mallya (NVIDIA)

[Ming-Yu Liu](/person/ming-yu-liu)

 

 

 ## Publication Date



Saturday, June 19, 2021

 

 ## Published in



[CVPR](https://cvpr2021.thecvf.com/)

 

 ## Research Area



[Artificial Intelligence and Machine Learning ](/research-area/machine-learning-artificial-intelligence)

[Computer Vision](/research-area/computer-vision)

 

 

 ## External Links



[Project page](https://nvlabs.github.io/face-vid2vid/)

[arXiv](https://arxiv.org/abs/2011.15126)

[Demo](http://nvidia-research-mingyuliu.com/vid2vid-cameo)

[Video](https://www.youtube.com/watch?v=nLYg9Waw72U)

 

 

 ## Copyright



This material is posted here with permission of the IEEE. Internal or personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution must be obtained from the IEEE by writing to <pubs-permissions@ieee.org>.