Democratizing Immersive Experiences with NVIDIA AI

Abstract

The AMRI team at NVIDIA Research presents novel immersive 3D experiences at GTC 2025 that allows users to move around in a streaming video, in 3D, in real-time. This allows for a highly immersive video viewing experience, especially when paired with 3D displays such as the Looking Glass Go or virtual/mixed reality headsets. To make free-viewpoint videos streamable, we contribute two key advancements: (a) an efficient, on-the-fly compression scheme for 3D dynamic videos, called QUEEN, and (b) a novel efficient rendering scheme for light field displays. QUEEN allows dynamic 3D videos to be compressed with high ratio and short training times, then rendered at over 300FPS, while our novel light field renderer is able to render multiple possible viewpoints (a light field quilt) in one efficient pass, running at >30FPS.