NVIDIA AI-Mediated Reality and Interaction Research Group

Welcome to the homepage of NVIDIA Research’s AI-Mediated Reality and Interaction (AMRI) Group, led by Dr. Shalini De Mello. Our mission is to create interactive physical AIs capable of operating in dynamic 4D worlds. We span fundamental research on the algorithms, theory and application of human-AI interaction and 4D world modeling using new ideas in AI, neural rendering, graphics, generative modeling, large language models, human behavior understanding and digital human creation.

We are always looking for motivated full-time research scientists and interns. Interested graduate students and experienced professionals should email the team lead or specific team members directly.

News

August 2026 - 3 new papers accepted to ECCV 2026!

July 2026 - 1 paper accepted to SIGGRAPH 2026 and 1 paper accepted to HPG 2026!

June 2026 - We are hosting 4 workshops at CVPR 2026!

December 2025 - 2 papers accepted to NeurIPS 2025! We are also presenting 1 Emerging Technologies demo at SIGGRAPH Asia 2025.

Members

Avatar

Shalini De Mello

Team Leader

Avatar

Alex Trevithick

Computer Vision, World Models, Machine Learning

Avatar

Amrita Mazumdar

Streaming, Compression, 3D Representations

Avatar

Christian Jacobsen

Computer Vision, Generative Modeling, Human Robot Interaction

Avatar

Haithem Turki

3D Reconstruction, Generative Models, Simulation

Avatar

Jonghyun Kim

AI display research, AR/VR glasses design, Holographic near-eye displays

Avatar

Koki Nagano

Digital Humans, 3D Generative Models, Media Forensics

Avatar

Michael Stengel

Computer Graphics, Telepresence

Avatar

Seonwook Park

Computer Vision, Perceiving Humans

Avatar

Shengze Wang

Lifelike Robots, Digital Humans, Telepresence

Avatar

Tianye Li

Digital Humans, Computer Vision, Graphics

Publications

Quickly discover relevant content by filtering publications.

COSY: Compositional 3DGS Synthesis for Disentangled Human Head Editing

European Conference on Computer Vision (ECCV), 2026

Recent 3D Gaussian Splatting (3DGS) GANs for human heads synthesize and render photorealistic 3D models in real-time and offer a vast variety in identity and appearance. However, controlling specific semantic attributes such as hair color or glasses …
Project arXiv Github

DETRAM: End-to-end DEtection, Tracking and Recovery of HumAn Meshes

European Conference on Computer Vision (ECCV), 2026

In the task of human mesh recovery (HMR), multi-person scenes are particularly difficult to handle due to the many entities that appear and occlusions between them over time. In particular for video inputs, there is a need to track each entity …
arXiv

Instant Expressive Gaussian Head Avatars at Over 100 FPS

European Conference on Computer Vision (ECCV), 2026

Portrait animation has witnessed tremendous quality improvements thanks to recent advances in video diffusion models. However, these 2D methods often compromise 3D consistency and speed, limiting their applicability in real-world scenarios, such as …
Project arXiv Video

Real-time 3D Visualization of Radiance Fields on Light Field Displays

High-Performance Graphics (HPG), 2026

Radiance fields, including their recent efficient forms such as 3D Gaussian Splatting and Sparse Voxels, have revolutionized photorealistic 3D scene visualization by enabling high-fidelity reconstruction of complex environments, making them a natural …
Project arXiv PDF Github

TSMC: Time-Varying Scene Mesh Compression

ACM SIGGRAPH, 2026

Time-varying scene meshes are a widely used representation for volumetric video, offering immersive six degrees of freedom (6DoF) interaction in virtual environments. However, their significantly larger size presents major challenges for efficient …
Project Github

VideoFDB: Evaluating Full-Duplex Vision-Speech Capabilities in Conversational Agents

ArXiv Preprint, 2026

Natural human conversation is full-duplex and audio-visual: people simultaneously speak and listen while continuously interpreting and producing nonverbal cues, such as nods, smiles, and gestures. To support successful human-agent interaction, agents …
Project arXiv PDF

Play4D: Accelerated and Interactive Free-viewpoint Video Streaming for Virtual Reality and Light Field Displays

ACM SIGGRAPH Asia 2025 Emerging Technologies, 2025

We present Play4D, a new method for accelerating and interactive free-viewpoint video streaming for virtual reality and light field displays. Our method is based on the observation that the face of a puppeteer is often visible in the background of …
Project PDF

Seeing What Matters: Generalizable AI-generated Video Detection with Forensic-Oriented Augmentation

Advances in Neural Information Processing Systems (NeurIPS), 2025

Synthetic video generation is progressing very rapidly. The latest models can produce very realistic high-resolution videos that are virtually indistinguishable from real ones. Although several video forensic detectors have been recently proposed, …
Project arXiv PDF Github

Unmasking Puppeteers: Leveraging Biometric Leakage to Expose Impersonation in AI-Based Videoconferencing

Advances in Neural Information Processing Systems (NeurIPS), 2025

AI-based videoconferencing systems have become increasingly popular, but they raise serious security concerns. Users may be impersonated by malicious actors, who can use AI-generated videos to deceive others. Existing methods for detecting such …
arXiv PDF

GeoMan: Temporally Consistent Human Geometry Estimation using Image-to-Video Diffusion

IEEE International Conference on Computer Vision (ICCV), 2025

Estimating accurate and temporally consistent 3D human geometry from videos is a challenging problem in computer vision. Existing methods, primarily optimized for single images, often suffer from temporal inconsistencies and fail to capture …
Project arXiv
See all publications