Our goal is to advance foundational technologies enabling AI systems to perceive, model, and meaningfully interact with the physical world. We have joined forces with the Dynamic Vision and Learning Group, the High-Fidelity Physics Research Group, the Applied Autonomous Vehicle Research Group, and were previously named the Toronto AI Lab.

NVIDIA Spatial Intelligence Lab (SIL)

Recent Releases

View all publications →

FlashDreams: High-performance inference and serving infra for interactive world models.

Code + Docs Released

Asset Harvester: Extracting 3D Assets from Autonomous Driving Logs for Simulation

Code + Models Released

AlpaSim: Modular Closed-Loop Simulator for Autonomous Vehicle Policies

Code + Dataset Released

NCore: Multi-Sensor Data Platform for Neural 3D Reconstruction

Library + Dataset Released

Kimodo: Scaling Controllable Human Motion Generation

Code + Models + Demo Released

OmniDreams: Generative Closed-Loop AV Simulation

GTC Demo

Physical AI in Jensen's Keynote at CES 2026

Cosmos Drive Demos

ChronoEdit: Temporal Reasoning for Image Editing and World Simulation

Code Released

fVDB-Reality-Capture: Scalable 3D Reconstruction

Library

Lyra: Generative 3D Scene Reconstruction

Code Released

Cosmos-Drive-Dreams: Synthetic Driving Data Generation

Code + Model + Dataset Released

CAL: Complete Anything in LiDAR

ICML 2025

VoMP: Volumetric Mechanical Property Fields

SIGGRAPH 2025

BTimer: Feed-Forward Bullet-Time Reconstruction

NeurIPS 2025

GEN3C: 3D-Informed Video Generation with Camera Control

CVPR 2025, Code Released

3DGUT: Gaussian Splatting with Distorted Cameras

CVPR 2025, Code Released

DiffusionRenderer: Neural Inverse and Forward Rendering

CVPR 2025, Code + Model Released

GTRS: Generalized Trajectory Scoring for Driving

CVPR 2025, Code Released

PartField: 3D Feature Fields for Part Segmentation

ICCV 2025, Code Released

ProtoMotions: Physics-Based Character Animation

Library

Kaolin: 3D Deep Learning Library for PyTorch

Library

Research Areas

WorldSimulation

Research on world modeling, autonomous vehicle and humanoid agents.

Physics Simulation

Engines for simulation of complex materials and dynamic systems.

4D Perception

Systems for learning on dynamic 3D scenes and spatiotemporal relations.

3D/4D ContentCreation & Editing

Generation, reconstruction, and editing of assets and scenes, creating dynamic 3D worlds.

Geometry Processing

Algorithms for processing and understanding 3D data.

SDKs, Libraries, & Tools

Spatial Data SDKs

ViPE and NCore enable video-based 3D perception and multi-sensor data management.

Spatial AI SDKs

Kaolin and fVDB enable 3D deep learning, reality capture, and sparse spatial AI.

Sparse Volume SDKs

Efficient methods for large-scale sparse volumetric datasets with minimal memory overhead.

Code Releases

Releases of implementations for our published research projects.

Applications

Human Motion Modeling

AI-driven systems for realistic human motion generation, interactive authoring, and physics-based control.

NuRec

Reconstruction service to create simulation environments from sensor data captured by autonomous vehicles.

OmniDreams

Real-time generative closed-loop autonomous vehicle simulation.