Isekai3D SDG: A Platform for Diverse Synthetic Gameplay Visual Data
We present Isekai3D SDG (Synthetic Data Generation), a platform for generating diverse 3D gameplay data for world models, computer vision, neural graphics, and gameplay agents. The platform combines reusable environments and characters with modular Unreal Engine plugins. A two-stage workflow records world simulation and then replays it to capture complementary views and rendering passes from the same underlying gameplay. Configurable cameras, resolutions, and frame rates support paired observations with frame-aligned camera, character, and action metadata. Outputs include RGB, depth, surface normals, and instance IDs. A packaged application supports automated data generation without installing the Unreal Engine editor or build tools, while custom plugins and compatible DLL extensions let researchers adapt individual components.
Citation
@misc{karp2026isekai3dsdg,
title = {{Isekai3D SDG}: A Platform for Diverse Synthetic Gameplay Visual Data},
author = {Rafal Karp and Gene Liu and Shih-Hsin Li and
Eric Courtemanche and Shengze Wang and Josef Spjut and
David Pankratz and Mischa Kolbe and Michael Stengel and
Sven Elflein and Adam Grygielski and Liwen Wu and
Anjul Patney},
year = {2026},
eprint = {2611.03626},
archivePrefix = {arXiv},
primaryClass = {cs.CV},
url = {https://arxiv.org/abs/2611.03626}
}