Adaptive Volumetric Mechanical Property Fields Invariant to Resolution
TL;DR: AdaVoMP generates high resolution fine-grained volumetric physics properties which can be used to produce interactive physically realistic worlds.
Opposed to VoMP v1, AdaVoMP can generate $16^3\times$ higher resolution properties, allow test-time scaling the resolution, and use a new generated adaptive volumetric structure.
Gallery
Interactive Simulatable Worlds
Hover to play
A Gaussian Splat + mesh scene turned into an interactive world using AdaVoMP, allowing a robot to interact with everything in the scene in a physically realistic way.
Gaussian Splat + Mesh scene
Simulating Meshes and Splats
Hover to play
Scenes with only meshes or with gaussian splats simulated in a physically realistic way using AdaVoMP.
Robot Evaluation
Hover to play
AdaVoMP can be used with RoboLab in Isaac Sim to aid in creating diverse interactive environments to benchmark any robot and any policy.
Physics Material Fields
Hover to play
Additional Physics Material Fields
Physics Material Fields and Structure Generated by AdaVoMP.
Abstract
Accurate mechanical properties (or materials) Young's modulus ($E$), Poisson's ratio ($\nu$) and density ($\rho$) are essential for reliable physics simulation of digital worlds, but most 3D assets lack this information. We propose AdaVoMP, a method for predicting accurate dense spatially-varying ($E$, $\nu$, $\rho$) for input 3D objects across representations, improving the resolution, accuracy, and memory efficiency over the state-of-the-art. The foundation of our technique is a sparse and adaptive voxel structure SAV that efficiently represents both the input 3D shape and the material field output. We replace the fixed-voxel model of the most accurate prior method, VoMP, with a novel sparse transformer encoder-decoder model that learns to generate a unique SAV autoregressively for every input shape to represent its materials, achieving a resolution $16^3\times$ higher than prior art. Experiments show that AdaVoMP estimates more accurate volumetric properties, even with lesser test-time compute than all prior art. This allows us to convert high-resolution complex 3D objects into simulation-ready assets, resulting in realistic deformable simulations.
Method
SAV: Sparse Adaptive Voxels
SAV is a sparse adaptive voxel representation that we use to encode both the input 3D shape and the output spatially varying materials. By efficiently representing geometry and materials in adaptive structures, we can allocate less compute to predict areas of piecewise constant materials, common in everyday objects (the wooden surface, the metal frame), while recursively refining only the fine heterogeneous regions and boundaries.
Adaptive Geometry Transformer & Adaptive Material Generator
Method Overview: input shape is encoded as AdaVoMP (top left), encoded (top right), and processed with our autoregressive Adaptive Material Generator (bottom), which is trained to output material field as SAV.
































