People

Shalini De Mello

Shalini De Mello is a Director of Research, New Experiences and a Distinguished Research Scientist at NVIDIA, where she leads the AI-Mediated Reality and Interaction Research Group. Previously, she was a researcher in the Learning and Perception Research Group at NVIDIA, from 2013 to 2023. Her research interests are in AI, computer vision and digital humans. Her research focuses on using AI to re-imagine interactions between humans, and between humans and machines. She has co-authored scores of peer-reviewed publications and patents. Her inventions have contributed to several NVIDIA AI products, including DriveIX, Maxine and TAO Toolkit. She received her Doctoral and Master’s degrees in Electrical and Computer Engineering from the University of Texas at Austin.

Complete Curriculum Vitae.

Research Area(s)
Artificial Intelligence and Machine Learning
Computer Vision
Generative AI
Human Computer Interaction
Main Field of Interest
Computer Vision
Google Scholar
https://scholar.google.com/citations?hl=en&user=xQM4BlMAAAAJ&cstart=20&view_op=…

Publications

2023

Convolutional State Space Models for Long-Range Spatiotemporal Modeling
Jimmy T. H. Smith, Shalini De Mello, Jan Kautz, Scott Linderman, Wonmin Byeon
Advances in Neural Information Processing Systems (NeurIPS) 2023
Generalizable One-shot 3D Neural Head Avatar
Xueting Li, Shalini De Mello, Sifei Liu, Koki Nagano, Umar Iqbal, Jan Kautz
Advances in Neural Information Processing Systems (NeurIPS) 2023
Generative Novel View Synthesis with 3D-Aware Diffusion Models
Eric R. Chan, Koki Nagano, Matthew Chan, Alexander W. Bergman, Jeong Joon Park, Axel Levy, Miika Aittala, Shalini De Mello, Tero Karras, Gordon Wetzstein
International Conference on Computer Vision (ICCV) 2023
Oral
AI-Mediated 3D Video Conferencing
Michael Stengel, Koki Nagano, Chao Liu, Matthew Chan, Alex Trevithick, Shalini De Mello, Jonghyun Kim, David Luebke, Amrita Mazumdar, Shengze Wang, Mayoore Jaiswal
ACM SIGGRAPH Emerging Technologies 2023
Affordance Diffusion: Synthesizing Hand-Object Interactions
Yufei Ye, Xueting Li, Abhinav Gupta, Shalini De Mello, Stan Birchfield, Jiaming Song, Shubham Tulsiani, Sifei Liu
IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2023
Open-Vocabulary Panoptic Segmentation with Text-to-Image Diffusion Models
Jiarui Xu, Sifei Liu, Arash Vahdat, Wonmin Byeon, Xiaolong Wang, Shalini De Mello
IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2023
Hightlight top 10%
Zero-shot Pose Transfer for Unrigged Stylized 3D Characters
Jiashun Wang, Xueting Li, Sifei Liu, Shalini De Mello, Orazio Gallo, Xiaolong Wang, Jan Kautz
IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2023
GazeNeRF: 3D-Aware Gaze Redirection with Neural Radiance Fields
Alessandro Ruzzi, Xiangwei Shi, Xi Wang, Gengyan Li, Shalini De Mello, Hyung Jin Chang, Xucong Zhang, Otmar Hilliges
IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2023
GPViT: A High Resolution Non-Hierarchical Vision Transformer with Group Propagation
Chenhongyi Yang, Jiarui Xu, Shalini De Mello, Elliot J. Crowley, Xiaolong Wang
International Conference on Learning Representations (ICLR) 2023
Notable top 25%, Oral

2022

CoordGAN: Self-Supervised Dense Correspondences Emerge from GANs
Jiteng Mu, Shalini De Mello, Zhiding Yu, Nuno Vasconcelos, Xiaolong Wang, Sifei Liu
IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2022
GroupViT: Semantic Segmentation Emerges from Text Supervision
Jiarui Xu, Shalini De Mello, Sifei Liu, Wonmin Byeon, Thomas Breuel, Jan Kautz, Xiaolong Wang
IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2022
Efficient Geometry-aware 3D Generative Adversarial Networks
Eric R. Chan, Connor Z. Lin, Matthew A. Chan, Koki Nagano, Boxiao Pan, Shalini De Mello, Orazio Gallo, Leonidas Guibas, Jonathan Tremblay, Sameh Khamis, Tero Karras, Gordon Wetzstein
IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2022
Oral
FreeSOLO: Learning to Segment Objects without Annotations
Xinlong Wang, Zhiding Yu, Shalini De Mello, Jan Kautz, Anima Anandkumar, Chunhua Shen, Jose M. Alvarez
IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2022
Learning Continuous Environment Fields via Implicit Functions
Xueting Li, Shalini De Mello, Xiaolong Wang, Ming-Hsuan Yang, Jan Kautz, Sifei Liu
International Conference on Learning Representations (ICLR), 2022
Learning Contrastive Representation for Semantic Correspondence
Taihong Xiao, Sifei Liu, Shalini De Mello, Zhiding Yu, Jan Kautz, Ming-Hsuan Yang
International Journal of Computer Vision (IJCV) 2022

2021

Self-Supervised Object Detection via Generative Image Synthesis
Siva Karthik Mustikovela, Shalini De Mello, Aayush Prakash, Umar Iqbal, Sifei Liu, Thu Nguyen-Phuoc, Carsten Rother, Jan Kautz
International Conference on Computer Vision (ICCV) 2021
Weakly-Supervised Physically Unconstrained Gaze Estimation
Rakshit Kothari, Shalini De Mello, Umar Iqbal, Wonmin Byeon, Seonwook Park, Jan Kautz
IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2021
Oral
Learning to Track Instances without Video Annotations
Yang Fu, Sifei Liu, Umar Iqbal, Shalini De Mello, Humphrey Shi, Jan Kautz
IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2021
Oral
Contrastive Syn-to-Real Generalization
Wuyang Chen, Zhiding Yu, Shalini De Mello, Sifei Liu, Jose M. Alvarez, Zhangyang Wang, Anima Anandkumar
International Conference on Learning Representations (ICLR) 2021

2020

Online Adaptation for Consistent Mesh Reconstruction in the Wild
Xueting Li, Sifei Liu, Shalini De Mello, Kihwan Kim, Xiaolong Wang, Ming-Hsuan Yang, Jan Kautz
Neural Information Processing Systems (NeurIPS) 2020
Self-Learning Transformations for Improving Gaze and Head Redirection
Yufeng Zheng, Seonwook Park, Xucong Zhang, Shalini De Mello, Otmar Hilliges
Neural Information Processing Systems (NeurIPS) 2020
Self-supervised Single-view 3D Reconstruction via Semantic Consistency
Xueting Li, Sifei Liu, Kihwan Kim, Shalini De Mello, Varun Jampani, Ming-Hsuan Yang, Jan Kautz
European Conference on Computer Vision (ECCV) 2020
Self-Supervised Viewpoint Learning From Image Collections
Siva Karthik Mustikovela, Varun Jampani, Shalini De Mello, Sifei Liu, Umar Iqbal, Carsten Rother, Jan Kautz
IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2020

2019

Joint-task Self-supervised Learning for Temporal Correspondence
Xueting Li, Sifei Liu, Shalini De Mello, Xiaolong Wang, Jan Kautz, Ming-Hsuan Yang
Neural Information Processing Systems (NeurIPS) 2019
Content-Consistent Generation of Realistic Eyes with Style
Marcel Bühler , Seonwook Park, Shalini De Mello, Xucong, Otmar Hilliges
International Conference on Computer Vision Workshop (ICCVW) 2019
Winner (1st place) Synthetic Eye Generation Challenge
Few-Shot Adaptive Gaze Estimation
Seonwook Park, Shalini De Mello, Pavlo Molchanov, Umar Iqbal, Otmar Hilliges, Jan Kautz
International Conference on Computer Vision (ICCV) 2019
Oral
Learning Propagation for Arbitrarily-Structured Data
Sifei Liu, Xueting Li, Varun Jampani, Shalini De Mello, Jan Kautz
International Conference on Computer Vision (ICCV) 2019
Few-Shot Viewpoint Estimation
Hung-Yu Tseng, Shalini De Mello, Jonathan Tremblay, Sifei Liu, Stan Birchfield, Ming-Hsuan Yang, Jan Kautz
British Machine Vision Conference (BMVC) 2019
NVGaze: An Anatomically-Informed Dataset for Low-Latency, Near-Eye Gaze Estimation
Joohwan Kim, Michael Stengel, Alexander Majercik, Shalini De Mello, David Dunn, Samuli Laine, Morgan McGuire, David Luebke
ACM Conference on Human-Computer-Interaction (CHI) 2019

2018

Switchable Temporal Propagation Network
Sifei Liu, Guangyu Zhong, Shalini De Mello, Jinwei Gu, Varun Jampani
European Conference on Computer Vision (ECCV) 2018
Light-weight Head Pose Invariant Gaze Tracking
Rajeev Ranjan, Shalini De Mello, Jan Kautz
IEEE Computer Vision and Pattern Recognition Workshop (CVPRW) 2018
Best Paper (runner up) Workshop on Analysis and Modeling of Faces and Gestures

2017

Learning Affinity via Spatial Propagation Networks
Sifei Liu, Shalini De Mello, Jinwei Gu, Guangyu Zhong, Ming-Hsuan Yang, Jan Kautz
Conference on Neural Information Processing Systems (NIPS) 2017
Dynamic Facial Analysis: From Bayesian Filtering to Recurrent Neural Network
Jinwei Gu, Xiaodong Yang, Shalini De Mello, Jan Kautz
IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2017

2016

Towards Selecting Robust Hand Gestures for Automotive Interfaces
Shalini Gupta, Pavlo Molchanov, Xiaodong Yang, Stephen Tyree, Jan Kautz
IEEE Intelligent Vehicles Symposium (IV) 2016
Online Detection and Classification of Dynamic Hand Gestures with Recurrent 3D Convolutional Neural Networks
Pavlo Molchanov, Xiaodong Yang, Shaline Gupta, Kihwan Kim, Stephen Tyree, Jan Kautz
IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2016

2015

Robust Model-based 3D Head Pose Estimation
Gregory P Meyer, Shalini Gupta, Iuri Frosio, Dikpal Reddy, Jan Kautz
IEEE International Conference on Computer Vision (ICCV) 2015
Hand Gesture Recognition with 3D Convolutional Neural Networks
Pavlo Molchanov, Shalini Gupta, Kihwan Kim, Jan Kautz
IEEE Computer Vision and Pattern Recognition Workshop (CVPRW) 2015
Winner (1st place) Hand Gesture Recognition Challenge
Multi-sensor System for Driver’s Hand-Gesture Recognition
Pavlo Molchanov, Shalini Gupta, Kihwan Kim, Kari Pulli
IEEE International Conference on Automatic Face and Gesture Recognition (FG) 20…