NVIDIA Logo Spatial Intelligence Lab

Instant NuRec:
Feed-Forward 3D Gaussian Reconstruction for Driving Scene Simulation

Research Jiahui Huang Jiawei Ren Michal Tyszkiewicz Xin Kang Seung Wook Kim Shengyu Huang Laura Leal-Taixe‡ Zan Gojcic‡ Sanja Fidler‡
Engineering Bjoern Haefner Michael Shelley Ning Xu Qi Wu Janick Martinez Esturo Nick Schneider‡

‡: Leadership

NVIDIA

📄 Paper Code 🤗 Model

Abstract

Instant NuRec ingests a multi-camera driving sequence and produces a layered 3DGS world with static, dynamic, and sky components in a single feed-forward pass.

3D simulation platforms are critical for autonomous driving because they enable end-to-end policy evaluation, thereby reducing development costs and improving safety. In recent years, neural simulation has become predominant, with methods such as NuRec playing a central role; however, these methods remain relatively slow and typically require per-scene tuning. In this work, we present Instant NuRec, a feed-forward neural reconstruction model that turns a short multi-view driving log into a fully simulatable 3D Gaussian Splatting (3DGS) world in a single forward pass. The model accepts multi-view input from a calibrated camera rig and emits a layered output consisting of static and dynamic 3DGS layers, a sky cubemap, and per-camera ISP corrections, while providing native support for non-pinhole camera models via 3DGUT. It reconstructs a 10–20-second multi-camera scene in roughly 1.5 seconds and achieves a PSNR on the Waymo Open Dataset that is 2.01 dB above the strongest evaluated baseline. Instant NuRec is deeply integrated into NuRec and is compatible with AlpaSim for closed-loop simulation.

Reconstruction Gallery

Bird's-eye-view reconstructions across diverse road layouts, weather conditions, and times of day.

Race Track Reconstruction

Long-clip reconstruction around a race track, shown from rendered and top-down viewpoints.

Closed-Loop Simulation

Closed-loop simulation rollouts in AlpaSim using Instant NuRec reconstructions.

Method

Multi-view driving images are tokenized into patches and processed by an alternating-attention ViT encoder. Several decoder heads share the resulting latent features and produce depth maps, semantic labels, motion estimates, a sky cubemap, and 3DGS attributes. Optionally, the output can be further optimized on a per-scene basis and used for downstream simulation tasks.

Posed multi-view input frames
Posed multi-view input frames Calibrated camera observations provide the input sequence.

Quantitative Results

Compare reconstruction quality and downstream policy evaluation results from the paper.

NuRec Instant NuRec - Dense Instant NuRec - Selective

PSNR (dB)

34.38 29.93 29.77
NuRecDenseSelective

Detection Precision

0.970 0.955 0.946
NuRecDenseSelective

Detection Recall

0.955 0.940 0.929
NuRecDenseSelective

Reconstruction Time (log seconds)

~75 min ~1.5 s ~1.5 s
NuRecDenseSelective
Select a bar to inspect its exact value, or use the legend to compare methods.

Instant NuRec reconstructs a scene in roughly 1.5 seconds while retaining strong image and detection quality.

Single-Camera Reconstruction
Single-camera Instant NuRec reconstruction shown from original, re-posed, and top-down views
From a single front-camera input, Instant NuRec produces geometrically consistent re-posed and top-down renderings.
LiDAR Reconstruction
Instant NuRec LiDAR reconstruction for recorded and shifted trajectories
The feed-forward LiDAR extension preserves dominant scene geometry under recorded and shifted trajectories.

Citation

@techreport{nvidia2026instantnurec, title = {Instant NuRec: Feed-Forward 3D Gaussian Reconstruction for Driving Scene Simulation}, author = {{NVIDIA}}, institution = {NVIDIA}, year = {2026} }

Acknowledgments

We greatly appreciate the contributions of the following individuals:

Alessandro Burzio, Alex Perec, Bingxin Ke, Daniel Dworakowski, Despoina Paschalidou, Emmanuel Attia, Jun Gao, Katarina Tothova, Lei Zhang, Lucrezia Shen, Murat Arar, Naveen Kumar Rai, Nicolas Moenne-Loccoz, Rodolfo Lima, Sangeetha Grama Srinivasan, Sean Pieper, Sergio Agostinho, Sherwin Bahmani, Shikhar Solanki, Sipeng Zhang, Tianchang Shen, Tobias Fischer, Weihua Zhang, Xuanchi Ren, Yixin Cao.