Sifei Liu ![](/sites/default/files/person/IMG_1516.JPG) Sifei Liu is a senior research scientist at Nvidia Research in Santa Clara, US. She received her PhD from the University of California Merced, department of EECS, where she was advised by Prof. [Ming-Hsuan Yang](http://faculty.ucmerced.edu/mhyang/). Before that, she obtained her master’s in ECE from University of Science and Technology of China (USTC), under the supervision of Prof. [Stan.Z Li ](http://www.nlpr.ia.ac.cn/users/szli/)and Prof. Bin Li, and bachelor’s in control science from North China Electric Power University (NCEPU). Her research interests are in computer vision (low-level vision, semantic segmentation, 3D scene understanding), deep learning (graph-structured networks, self-supervised learning), and the combination of both. She also worked as an intern student in Baidu IDL, multimedia lab in CUHK, and NVIDIA research. She was the recipient of Baidu Graduate Fellowship in 2013, and NVIDIA Pioneering Research Award in 2017. Research Area(s) [Artificial Intelligence and Machine Learning ](/research-area/machine-learning-artificial-intelligence) [Generative AI](/research-area/generative-ai) Main Field of Interest [Computer Vision](/research-area/computer-vision) Google Scholar [https://scholar.google.com/citations?user=j4pcHV4AAAAJ&hl=en](https://scholar.google.com/citations?user=j4pcHV4AAAAJ&hl=en) ### Publications ### 2024 [CosAE: Learnable Fourier Series for Image Restoration](/publication/2024-12_cosae-learnable-fourier-series-image-restoration) [Sifei Liu](/person/sifei-liu), [Shalini De Mello](/person/shalini-de-mello), [Jan Kautz](/person/jan-kautz) [Advances in Neural Information Processing Systems (NeurIPS) 2024](https://proceedings.neurips.cc/paper_files/paper/2024/file/13e8be77982beb73d7ed0bbf122f9f3c-Paper-Conference.pdf) [Dream-in-4D: A Unified Approach for Text- and Image-guided 4D Scene Generation](/publication/2024-06_dream-4d-unified-approach-text-and-image-guided-4d-scene-generation) Yufeng Zheng, [Xueting Li](/person/xueting-li), [Koki Nagano](/person/koki-nagano), [Sifei Liu](/person/sifei-liu), Otmar Hilliges, [Shalini De Mello](/person/shalini-de-mello) [IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2024](https://openaccess.thecvf.com/content/CVPR2024/papers/Zheng_A_Unified_Approach_for_Text-_and_Image-guided_4D_Scene_Generation_CVPR_2024_paper.pdf) [RegionGPT: Towards Region Understanding Vision Language Model](/publication/2024-06_regiongpt-towards-region-understanding-vision-language-model) Qiushan Guo, [Shalini De Mello](/person/shalini-de-mello), [Hongxu Danny Yin](/person/danny-yin), [Wonmin Byeon](/person/wonmin-byeon), Ka Chun Cheung, Yizhou Yu, Ping Luo, [Sifei Liu](/person/sifei-liu) [IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2024](https://openaccess.thecvf.com/content/CVPR2024/papers/Guo_RegionGPT_Towards_Region_Understanding_Vision_Language_Model_CVPR_2024_paper.pdf) [3D Reconstruction with Generalizable Neural Fields using Scene Priors](/index.php/publication/2024-05_3d-reconstruction-generalizable-neural-fields-using-scene-priors) Yang Fu, [Shalini De Mello](/index.php/person/shalini-de-mello), [Xueting Li](/index.php/person/xueting-li), Amey Kulkarni, [Jan Kautz](/index.php/person/jan-kautz), Xiaolong Wang, [Sifei Liu](/index.php/person/sifei-liu) [International Conference on Learning Representations (ICLR) 2024](https://proceedings.iclr.cc/paper_files/paper/2024/hash/0bd32794b26cfc99214b89313764da8e-Abstract-Conference.html) ### 2023 [Generalizable One-shot 3D Neural Head Avatar](/publication/2023-12_generalizable-one-shot-3d-neural-head-avatar) [Xueting Li](/person/xueting-li), [Shalini De Mello](/person/shalini-de-mello), [Sifei Liu](/person/sifei-liu), [Koki Nagano](/person/koki-nagano), [Umar Iqbal](/person/umar-iqbal), [Jan Kautz](/person/jan-kautz) [Advances in Neural Information Processing Systems (NeurIPS) 2023](https://neurips.cc/) [Affordance Diffusion: Synthesizing Hand-Object Interactions](/publication/2023-06_affordance-diffusion-synthesizing-hand-object-interactions) Yufei Ye, [Xueting Li](/person/xueting-li), Abhinav Gupta, [Shalini De Mello](/person/shalini-de-mello), [Stan Birchfield](/person/stan-birchfield), Jiaming Song, Shubham Tulsiani, [Sifei Liu](/person/sifei-liu) [IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2023](https://cvpr2023.thecvf.com/) [Open-Vocabulary Panoptic Segmentation with Text-to-Image Diffusion Models](/publication/2023-06_open-vocabulary-panoptic-segmentation-text-image-diffusion-models) Jiarui Xu, [Sifei Liu](/person/sifei-liu), [Arash Vahdat](/person/arash-vahdat), [Wonmin Byeon](/person/wonmin-byeon), Xiaolong Wang, [Shalini De Mello](/person/shalini-de-mello) [IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2023](https://cvpr2023.thecvf.com/) Hightlight top 10% [Zero-shot Pose Transfer for Unrigged Stylized 3D Characters](/publication/2023-06_zero-shot-pose-transfer-unrigged-stylized-3d-characters) Jiashun Wang, [Xueting Li](/person/xueting-li), [Sifei Liu](/person/sifei-liu), [Shalini De Mello](/person/shalini-de-mello), Orazio Gallo, Xiaolong Wang, [Jan Kautz](/person/jan-kautz) [IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2023](https://openaccess.thecvf.com/content/CVPR2023/papers/Wang_Zero-Shot_Pose_Transfer_for_Unrigged_Stylized_3D_Characters_CVPR_2023_paper.pdf) ### 2022 [CoordGAN: Self-Supervised Dense Correspondences Emerge from GANs](/publication/2022-07_coordgan-self-supervised-dense-correspondences-emerge-gans) Jiteng Mu, [Shalini De Mello](/person/shalini-de-mello), [Zhiding Yu](/person/zhiding-yu), Nuno Vasconcelos, Xiaolong Wang, [Sifei Liu](/person/sifei-liu) [IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2022](https://cvpr2022.thecvf.com/) [GroupViT: Semantic Segmentation Emerges from Text Supervision](/publication/2022-06_groupvit-semantic-segmentation-emerges-text-supervision) Jiarui Xu, [Shalini De Mello](/person/shalini-de-mello), [Sifei Liu](/person/sifei-liu), [Wonmin Byeon](/person/wonmin-byeon), [Thomas Breuel](/person/thomas-breuel), [Jan Kautz](/person/jan-kautz), Xiaolong Wang [IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2022](https://cvpr2022.thecvf.com/) [Learning Continuous Environment Fields via Implicit Functions](/index.php/publication/2022-04_learning-continuous-environment-fields-implicit-functions) [Xueting Li](/index.php/person/xueting-li), [Shalini De Mello](/index.php/person/shalini-de-mello), Xiaolong Wang, Ming-Hsuan Yang, [Jan Kautz](/index.php/person/jan-kautz), [Sifei Liu](/index.php/person/sifei-liu) [International Conference on Learning Representations (ICLR), 2022](https://iclr.cc/Conferences/2022) [Learning Contrastive Representation for Semantic Correspondence](/index.php/publication/2022-03_learning-contrastive-representation-semantic-correspondence) Taihong Xiao, [Sifei Liu](/index.php/person/sifei-liu), [Shalini De Mello](/index.php/person/shalini-de-mello), [Zhiding Yu](/index.php/person/zhiding-yu), [Jan Kautz](/index.php/person/jan-kautz), Ming-Hsuan Yang [International Journal of Computer Vision (IJCV) 2022](https://link.springer.com/article/10.1007/s11263-022-01602-y) ### 2021 [Self-Supervised Object Detection via Generative Image Synthesis](/index.php/publication/2021-10_self-supervised-object-detection-generative-image-synthesis) Siva Karthik Mustikovela, [Shalini De Mello](/index.php/person/shalini-de-mello), Aayush Prakash, [Umar Iqbal](/index.php/person/umar-iqbal), [Sifei Liu](/index.php/person/sifei-liu), Thu Nguyen-Phuoc, Carsten Rother, [Jan Kautz](/index.php/person/jan-kautz) [International Conference on Computer Vision (ICCV) 2021](https://iccv2021.thecvf.com/) [Learning to Track Instances without Video Annotations](/index.php/publication/2021-06_learning-track-instances-without-video-annotations) Yang Fu, [Sifei Liu](/index.php/person/sifei-liu), [Umar Iqbal](/index.php/person/umar-iqbal), [Shalini De Mello](/index.php/person/shalini-de-mello), Humphrey Shi, [Jan Kautz](/index.php/person/jan-kautz) [IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2021](http://cvpr2021.thecvf.com/) Oral [Contrastive Syn-to-Real Generalization](/publication/2021-05_contrastive-syn-real-generalization) Wuyang Chen, [Zhiding Yu](/person/zhiding-yu), [Shalini De Mello](/person/shalini-de-mello), [Sifei Liu](/person/sifei-liu), Jose M. Alvarez, Zhangyang Wang, Anima Anandkumar [International Conference on Learning Representations (ICLR) 2021](https://openreview.net/group?id=ICLR.cc/2021/Conference) ### 2020 [Online Adaptation for Consistent Mesh Reconstruction in the Wild](/index.php/publication/2020-12_online-adaptation-consistent-mesh-reconstruction-wild) Xueting Li, [Sifei Liu](/index.php/person/sifei-liu), [Shalini De Mello](/index.php/person/shalini-de-mello), Kihwan Kim, Xiaolong Wang, Ming-Hsuan Yang, [Jan Kautz](/index.php/person/jan-kautz) [Neural Information Processing Systems (NeurIPS) 2020](https://nips.cc/virtual/2020/public/poster_aba3b6fd5d186d28e06ff97135cade7f.html) [Self-supervised Single-view 3D Reconstruction via Semantic Consistency](/index.php/publication/2020-08_self-supervised-single-view-3d-reconstruction-semantic-consistency) Xueting Li, [Sifei Liu](/index.php/person/sifei-liu), Kihwan Kim, [Shalini De Mello](/index.php/person/shalini-de-mello), Varun Jampani, Ming-Hsuan Yang, [Jan Kautz](/index.php/person/jan-kautz) [European Conference on Computer Vision (ECCV) 2020](https://eccv2020.eu/) [Self-Supervised Viewpoint Learning From Image Collections ](/index.php/publication/2020-06_self-supervised-viewpoint-learning-image-collections) Siva Karthik Mustikovela, Varun Jampani, [Shalini De Mello](/index.php/person/shalini-de-mello), [Sifei Liu](/index.php/person/sifei-liu), [Umar Iqbal](/index.php/person/umar-iqbal), Carsten Rother, [Jan Kautz](/index.php/person/jan-kautz) [IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2020](http://cvpr2020.thecvf.com/) ### 2019 [Joint-task Self-supervised Learning for Temporal Correspondence ](/index.php/publication/2019-12_joint-task-self-supervised-learning-temporal-correspondence) Xueting Li, [Sifei Liu](/index.php/person/sifei-liu), [Shalini De Mello](/index.php/person/shalini-de-mello), Xiaolong Wang, [Jan Kautz](/index.php/person/jan-kautz), Ming-Hsuan Yang [Neural Information Processing Systems (NeurIPS) 2019](https://sites.google.com/view/uvc2019) [Learning Propagation for Arbitrarily-Structured Data](/index.php/publication/2019-09_learning-propagation-arbitrarily-structured-data) [Sifei Liu](/index.php/person/sifei-liu), Xueting Li, Varun Jampani, [Shalini De Mello](/index.php/person/shalini-de-mello), [Jan Kautz](/index.php/person/jan-kautz) [International Conference on Computer Vision (ICCV) 2019](http://openaccess.thecvf.com/content_ICCV_2019/html/Liu_Learning_Propagation_for_Arbitrarily-Structured_Data_ICCV_2019_paper.htm) [Few-Shot Viewpoint Estimation](/publication/2019-09_few-shot-viewpoint-estimation) Hung-Yu Tseng, [Shalini De Mello](/person/shalini-de-mello), [Jonathan Tremblay](/person/jonathan-tremblay), [Sifei Liu](/person/sifei-liu), [Stan Birchfield](/person/stan-birchfield), Ming-Hsuan Yang, [Jan Kautz](/person/jan-kautz) [British Machine Vision Conference (BMVC) 2019](https://bmvc2019.org/programme/) [SCOPS: Self-Supervised Co-Part Segmentation](/publication/2019-06_scops-self-supervised-co-part-segmentation) Wei-Chih Hung, Varun Jampani, [Sifei Liu](/person/sifei-liu), [Pavlo Molchanov](/person/pavlo-molchanov), Ming-Hsuan Yang , [Jan Kautz](/person/jan-kautz) [CVPR 2019](https://varunjampani.github.io/scops/) [Putting Humans in a Scene: Learning Affordance in 3D Indoor Environments](/index.php/publication/2019-06_putting-humans-scene-learning-affordance-3d-indoor-environments) Xueting Li, [Sifei Liu](/index.php/person/sifei-liu), Kihwan Kim, Xiaolong Wang, Ming-Hsuan Yang, [Jan Kautz](/index.php/person/jan-kautz) [CVPR 2019](https://sites.google.com/view/3d-affordance-cvpr19) [Learning Linear Transformations for Fast Image and Video Style Transfer](/publication/2019-04_learning-linear-transformations-fast-image-and-video-style-transfer) Xueting Li, [Sifei Liu](/person/sifei-liu), Ming-Hsuan Yang, [Jan Kautz](/person/jan-kautz) [CVPR 2019](https://sites.google.com/view/linear-style-transfer-cvpr19) ### 2018 [Context-aware Synthesis and Placement of Object Instances](/index.php/publication/2018-12_context-aware-synthesis-and-placement-object-instances) Donghoon Lee, [Sifei Liu](/index.php/person/sifei-liu), Jinwei Gu, [Ming-Yu Liu](/index.php/person/ming-yu-liu), Ming-Hsuan Yang, [Jan Kautz](/index.php/person/jan-kautz) NIPS [Switchable Temporal Propagation Network ](/publication/2018-09_switchable-temporal-propagation-network) [Sifei Liu](/person/sifei-liu), Guangyu Zhong, [Shalini De Mello](/person/shalini-de-mello), [Jinwei Gu](/person/jinwei-gu), Varun Jampani [European Conference on Computer Vision (ECCV) 2018](http://faculty.ucmerced.edu/mhyang/papers/eccv2018_stpn.pdf) ### 2017 [Learning Affinity via Spatial Propagation Networks ](/index.php/publication/2017-12_learning-affinity-spatial-propagation-networks) [Sifei Liu](/index.php/person/sifei-liu), [Shalini De Mello](/index.php/person/shalini-de-mello), [Jinwei Gu](/index.php/person/jinwei-gu), Guangyu Zhong, Ming-Hsuan Yang, [Jan Kautz](/index.php/person/jan-kautz) [Conference on Neural Information Processing Systems (NIPS) 2017](https://nips.cc/)