Abstract
We present Play4D, an accelerated and interactive free-viewpoint video (FVV) streaming pipeline for next-generation light field (LF) and virtual reality (VR) displays. Play4D unlocks efficient dynamic 4D neural scene reconstruction for Physical AI testing in Isaac Sim, along with their compressed streaming and free viewpoint immersive viewing on VR and light field displays. Our demo showcases two complementary experiences: Play4D-LF, a walk-up station offering interactive multi-view playback on a LF display; and Play4D-VR, a headset-based experience that enables full volumetric navigation through 4DGS content.
Play4D-VR: Interactive Real-to-Sim 4DGS in Virtual Reality
Interactive Real-to-Sim 4DGS in Virtual Reality.
Our Play4D-VR demo allows users to step into a 4D dynamic real-to-sim world where Physical AIs are tested in bespoke captured environments. Our system brings QUEEN-encoded 4DGS scenes into Isaac Lab, enabling robot-policy evaluation with free-viewpoint observation throughout the scene.
Streaming 3DGS Free-Viewpoint Video with QUEEN
(a) Boxer (ClearAngle / Prism.AI)
(b) Family (Infinite Realities)
In this demo, we use QUEEN to compress a dynamic 3D scene into a quantized set of 3D Gaussian primitives, then play the reconstructed 4DGS content back with free-viewpoint interaction. Our on-the-fly quantization-and-sparsity QUEEN encoder achieves up to 20× compression while preserving visual quality.
G2LF: Interactive Multi-view Playback on a Light Field Display
Our G2LF demo shows 4DGS scenes on a light-field display with immersive multi-view navigation. The single-pass, plane-sweeping G2LF renderer delivers up to 20× faster light-field rendering while preserving high visual quality
Partners
We thank our partners for providing the high quality 3D and 4D content we use for our demo.
Citation
@inproceedings{kim2025play4d,
author = {Kim, Jonghyun and Stengel, Michael and Mazumdar, Amrita and Li, Tianye and Sun, Cheng and Luebke, David and De Mello, Shalini},
title = {Play4D: Accelerated and Interactive Free-viewpoint Video Streaming for Virtual Reality and Light Field Displays},
booktitle = {SIGGRAPH Asia 2025 Emerging Technologies},
month = {December},
year = {2025}
}This website is based off of the EG3D website.