Toolkits for managing, processing, and analyzing large-scale spatial datasets across diverse domains.
Accurate 3D geometric perception is an important prerequisite for a wide range of spatial AI systems. However, acquiring consistent and precise 3D annotations from in-the-wild videos remains a key challenge. In this work, we introduce ViPE, a fast and versatile video processing engine designed to bridge this gap. ViPE efficiently estimates camera intrinsics, camera motion, and dense, near-metric depth maps from unconstrained raw videos. It is robust to diverse scenarios, including dynamic selfie videos, cinematic shots, or dashcams, and supports various camera models such as pinhole, wide-angle, and 360° panoramas.
We use ViPE to annotate a large-scale collection of videos. In total, the collection contains approximately 96M frames – all annotated with accurate camera poses and dense depth maps. We hope this dataset will help accelerate the development of spatial AI systems.
Camera Intrinsics Estimation • Camera Pose & Motion Estimation • Dense Near-Metric Depth Maps • Pinhole / Wide-Angle / 360° Panorama Support • Robust to Dynamic Scenes • 96M-Frame Annotated Video Collection
Source code, install instructions, and examples. Apache 2.0 license.
Overview, demos, and additional details.
Installation, CLI usage, examples, and dataset documentation.
Method, benchmarks against TUM/KITTI, and dataset analysis.
NVIDIA NCore is an open-source Python library that provides a canonical data representation for multi-sensor recordings—cameras, lidars, radars, poses, calibrations, and labels. It defines a modular, component-based format with rigorous coordinate conventions, GPU-accelerated sensor models on PyTorch/CUDA, and a streamable, cloud-native .itar storage format designed for distributed training at scale. NCore is the data backbone for NVIDIA NuRec and is integrated into reconstruction engines like 3DGRUT and gsplat.
Modular Component-Based Data Format • GPU-Accelerated Camera & Lidar Sensor Models • Rolling-Shutter-Aware Projection • Streamable Cloud-Native .itar Storage • Flexible Pose Graph (Rigged & Free-Posed) • Non-Redundant Storage Design • Built-in Dataset Converters • Interactive 3D Viewer • Strongly Typed Data Access APIs
Source code, issues, and contributions. Apache 2.0 license. Install with pip install nvidia-ncore.
Data format specification, API reference, tutorials, sensor model documentation, and tool guides.
Overview, architecture, design principles, benchmarks, and quickstart guide.