Spatial Intelligence Lab
Recent 3D generative models produce high-quality geometry from a single image using large-scale priors and diffusion architectures. However, they assume complete visibility and single-view inputs, limiting applicability in multi-view, occluded, or editing scenarios. Although prior works address these challenges individually, they lack a unified framework for controllable 3D completion under diverse conditioning signals. We present Axolotl3D, a multi-modal and occlusion-aware 3D generation model that jointly conditions on images, visibility masks, camera parameters, and a partial point cloud. The point cloud serves as a geometric anchor promoting faithful shape completion, while camera parameters ensure consistent multi-view alignment in a shared 3D coordinate system. A unified training strategy synthesizes diverse conditioning regimes from large-scale 3D data, enabling robust cross-modal reasoning. Experiments on Toys4K and OmniObject3D demonstrate state-of-the-art performance under both clean and occluded settings, as well as strong results in real-world reconstruction and geometry-consistent editing.
This video features Kaolin's new web UI and simulation features, demonstrated through interactive shape editing, splat segmentation to mesh, and real-world scene simulation.
Given multi-modal inputs—posed images, visibility masks, and partial points—our method encodes each modality and fuses them into multi-modal condition tokens. We fine-tune Hunyuan3D-DiT on these tokens to produce completed shape latents, which are decoded by the ShapeVAE.
To enable robust cross-modal reasoning, we propose a unified training strategy that synthesizes diverse conditioning regimes on-the-fly from large-scale 3D mesh datasets. By simulating partial observations, occlusions, and mixed image–geometry inputs, we train the model to generalize across single-view, sparse multi-view, and editing scenarios.
We evaluate our approach on 3D object datasets including Toys4K and OmniObject3D under both single-view and multi-view settings, with and without synthetic occlusions. Across all scenarios, Axolotl3D outperforms state-of-the-art methods in geometric accuracy and reconstruction fidelity.
We show examples under increasing amounts of occlusion. Our approach effectively handles severe occlusions by leveraging information from geometrically similar, unoccluded regions.
Shape editing conditioned on inpainted single-view image. Our method utilizes conditioning points to remain faithful to unedited areas.
Axolotl3D is robust to sparse and noisy point inputs, allowing it to leverage camera and point predictions from Pi3X to produce faithful 3D generations from a single view or sparse views.
Axolotl3D accurately completes partially observed objects in captured scenes for more complete and faithful physical simulation.
@inproceedings{hu2026axolotl3d,
title={Axolotl3D: a Unified Framework for Faithful 3D Shape Completion},
author={Hu, Anita and Shugrina, Maria},
booktitle={European Conference on Computer Vision (ECCV)},
year={2026}
}