NVIDIA AI-Mediated Reality and Interaction Research Group

Welcome to the homepage of NVIDIA’s AI-Mediated Reality and Interaction Research Group within the larger NVIDIA Research orgnization, led by Dr. Shalini De Mello. Our mission is to re-imagine, using AI, communication between humans, and between humans and machines. Our research spans AI, vision and graphics. We are particularly interested in:

  • Digital Humans: generative models for digital humans
  • Human Understanding: embodied perception of human state and behavior
  • Core AI Technologies: neural rendering, generative AI, streaming and displays
  • New Experiences: enabled by AI (3D video conferencing, content creation, XR, LLMs, etc.)

We are looking for motivated full-time research scientists and interns! Interested graduate students please email the team lead or specific members directly.

News

July 2025 - 1 paper accepted to SIGGRAPH 2025 and 1 paper accepted to ICCV 2025. We will also be presenting 2 live demos at SIGGRAPH 2025.

March 2025 - We presented a demo on Democratizing Immersive Experiences with NVIDIA AI at GTC 2025, showcasing our research innovations.

February 2025 - 3 new papers accepted to CVPR 2025! We are also organizing 2 workshops and 1 tutorial.

Members

Avatar

Shalini De Mello

Team Leader

Avatar

Amrita Mazumdar

Streaming, Compression, 3D Representations

Avatar

Jonghyun Kim

AI display research, AR/VR glasses design, Holographic near-eye displays

Avatar

Koki Nagano

Digital Humans, 3D Generative Models, Media Forensics

Avatar

Michael Stengel

Computer Graphics, Telepresence

Avatar

Omer Shapira

XR, Graphics, Human-Computer Interaction

Avatar

Seonwook Park

Computer Vision, Perceiving Humans

Avatar

Tianye Li

Digital Humans, Computer Vision, Graphics

Publications

Quickly discover relevant content by filtering publications.

GeoMan: Temporally Consistent Human Geometry Estimation using Image-to-Video Diffusion

IEEE International Conference on Computer Vision (ICCV), 2025

Estimating accurate and temporally consistent 3D human geometry from videos is a challenging problem in computer vision. Existing methods, primarily optimized for single images, often suffer from temporal inconsistencies and fail to capture …
Project arXiv

GAIA: Generative Animatable Interactive Avatars with Expression-conditioned Gaussians

SIGGRAPH (Conference Track), 2025

3D generative models of faces trained on in-the-wild image collections have improved greatly in recent times, offering better visual fidelity and view consistency. Making such generative models animatable is a hard yet rewarding task, with …
Project PDF Video

BLADE: Single-view Body Mesh Learning through Accurate Depth Estimation

IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2025

Single-image human mesh recovery is a challenging task due to the ill-posed nature of simultaneous body shape, pose, and camera estimation. Existing estimators work well on images taken from afar, but they break down as the person moves close to the …
Project arXiv PDF Video

Coherent 3D Portrait Video Reconstruction via Triplane Fusion

IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2025

Recent breakthroughs in single-image 3D portrait reconstruction have enabled telepresence systems to stream 3D portrait videos from a single camera in real-time, potentially democratizing telepresence. However, per-frame 3D reconstruction exhibits …
Project arXiv PDF

SimAvatar: Simulation-Ready Clothed Gaussian Avatars from Text

IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2025

We introduce SimAvatar, a framework designed to generate simulation-ready clothed 3D human avatars from a text prompt. Current text-driven human avatar generation methods either model hair, clothing, and the human body using a unified geometry or …
Project arXiv PDF Video

Democratizing Immersive Experiences with NVIDIA AI

GPU Technology Conference (GTC), 2025

The AMRI team at NVIDIA Research presents novel immersive 3D experiences at GTC 2025 that allows users to move around in a streaming video, in 3D, in real-time. This allows for a highly immersive video viewing experience, especially when paired with …
Project

CosAE: Learnable Fourier Series for Image Restoration

Advances in Neural Information Processing Systems (NeurIPS), 2024

In this paper, we introduce Cosine Autoencoder (CosAE), a novel, generic Autoencoder that seamlessly leverages the classic Fourier series with a feed-forward neural network. CosAE represents an input image as a series of 2D Cosine time series, each …
Project PDF Video

Large Étendue 3D Holographic Display with Content-adpative Dynamic Fourier Modulation

SIGGRAPH ASIA (Conference Track), 2024

Emerging holographic display technology offers unique capabilities for next-generation virtual reality systems. Current holographic near-eye displays, however, only support a small étendue, which results in a direct tradeoff between achievable field …
Project PDF Video

QUEEN: QUantized Efficient ENcoding for Streaming Free-viewpoint Videos

Advances in Neural Information Processing Systems (NeurIPS), 2024

Online free-viewpoint video (FVV) streaming is a challenging problem, which is relatively under-explored. It requires incremental on-the-fly updates to a volumetric representation, fast training and rendering to satisfy real-time constraints and a …
Project arXiv PDF Blog

Avatar Fingerprinting for Authorized Use of Synthetic Talking-Head Videos

European Conference on Computer Vision (ECCV), 2024

Modern generators render talking-head videos with impressive photorealism, ushering in new user experiences such as videoconferencing under constrained bandwidth budgets. Their safe adoption, however, requires a mechanism to verify if the rendered …
Project arXiv PDF Video
See all publications