NVIDIA AI-Mediated Reality and Interaction Research Group

Welcome to the homepage of NVIDIA Research’s AI-Mediated Reality and Interaction (AMRI) Group, led by Dr. Shalini De Mello. Our mission is to create interactive physical AIs capable of operating in dynamic 4D worlds. We span fundamental research on the algorithms, theory and application of human-AI interaction and 4D world modeling using new ideas in AI, neural rendering, graphics, generative modeling, large language models, human behavior understanding and digital human creation.

We are looking for motivated fulltime Senior Research Scientists, Research Scientists and Interns for 2026! Interested graduate students and experienced professionals should email the team lead or specific team members directly.

News

December 2025 - 2 papers accepted to NeurIPS 2025! We are also presenting 1 Emerging Technologies demo at SIGGRAPH Asia 2025.

July 2025 - 1 paper accepted to SIGGRAPH 2025 and 1 paper accepted to ICCV 2025! We also presenting 2 live demos at SIGGRAPH 2025 and organizing 1 workshop at ICCV 2025.

February 2025 - 3 new papers accepted to CVPR 2025! We are also organizing 2 workshops and 1 tutorial.

Members

Avatar

Shalini De Mello

Team Leader

Avatar

Alex Trevithick

Computer Vision, World Models, Machine Learning

Avatar

Amrita Mazumdar

Streaming, Compression, 3D Representations

Avatar

Jonghyun Kim

AI display research, AR/VR glasses design, Holographic near-eye displays

Avatar

Koki Nagano

Digital Humans, 3D Generative Models, Media Forensics

Avatar

Michael Stengel

Computer Graphics, Telepresence

Avatar

Seonwook Park

Computer Vision, Perceiving Humans

Avatar

Shengze Wang

Lifelike Robots, Digital Humans, Telepresence

Avatar

Tianye Li

Digital Humans, Computer Vision, Graphics

Publications

Quickly discover relevant content by filtering publications.

Play4D: Accelerated and Interactive Free-viewpoint Video Streaming for Virtual Reality and Light Field Displays

ACM SIGGRAPH Asia 2025 Emerging Technologies, 2025

We present Play4D, a new method for accelerating and interactive free-viewpoint video streaming for virtual reality and light field displays. Our method is based on the observation that the face of a puppeteer is often visible in the background of …
PDF

Seeing What Matters: Generalizable AI-generated Video Detection with Forensic-Oriented Augmentation

Advances in Neural Information Processing Systems (NeurIPS), 2025

Synthetic video generation is progressing very rapidly. The latest models can produce very realistic high-resolution videos that are virtually indistinguishable from real ones. Although several video forensic detectors have been recently proposed, …
Project arXiv PDF Github

Unmasking Puppeteers: Leveraging Biometric Leakage to Expose Impersonation in AI-Based Videoconferencing

Advances in Neural Information Processing Systems (NeurIPS), 2025

AI-based videoconferencing systems have become increasingly popular, but they raise serious security concerns. Users may be impersonated by malicious actors, who can use AI-generated videos to deceive others. Existing methods for detecting such …
arXiv PDF

GeoMan: Temporally Consistent Human Geometry Estimation using Image-to-Video Diffusion

IEEE International Conference on Computer Vision (ICCV), 2025

Estimating accurate and temporally consistent 3D human geometry from videos is a challenging problem in computer vision. Existing methods, primarily optimized for single images, often suffer from temporal inconsistencies and fail to capture …
Project arXiv

GAIA: Generative Animatable Interactive Avatars with Expression-conditioned Gaussians

SIGGRAPH (Conference Track), 2025

3D generative models of faces trained on in-the-wild image collections have improved greatly in recent times, offering better visual fidelity and view consistency. Making such generative models animatable is a hard yet rewarding task, with …
Project PDF Video

Real-time 3D Visualization of Radiance Fields on Light Field Displays

ArXiv Preprint, 2025

We present a real-time 3D visualization system for radiance fields on light field displays. Our system is able to render 3D scenes from a radiance field in real-time, and it is able to handle dynamic scenes with high frame rates. Our system is able …
arXiv PDF

BLADE: Single-view Body Mesh Learning through Accurate Depth Estimation

IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2025

Single-image human mesh recovery is a challenging task due to the ill-posed nature of simultaneous body shape, pose, and camera estimation. Existing estimators work well on images taken from afar, but they break down as the person moves close to the …
Project arXiv PDF Video Github

Coherent 3D Portrait Video Reconstruction via Triplane Fusion

IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2025

Recent breakthroughs in single-image 3D portrait reconstruction have enabled telepresence systems to stream 3D portrait videos from a single camera in real-time, potentially democratizing telepresence. However, per-frame 3D reconstruction exhibits …
Project arXiv PDF

SimAvatar: Simulation-Ready Clothed Gaussian Avatars from Text

IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2025

We introduce SimAvatar, a framework designed to generate simulation-ready clothed 3D human avatars from a text prompt. Current text-driven human avatar generation methods either model hair, clothing, and the human body using a unified geometry or …
Project arXiv PDF Video

Democratizing Immersive Experiences with NVIDIA AI

GPU Technology Conference (GTC), 2025

The AMRI team at NVIDIA Research presents novel immersive 3D experiences at GTC 2025 that allows users to move around in a streaming video, in 3D, in real-time. This allows for a highly immersive video viewing experience, especially when paired with …
Project
See all publications