PlaMo: Plan and Move in Rich 3D Physical Environments

PlaMo animates a humanoid in complex 3D scenes based on a sequence of natural language instructions describing landmarks and locomotion styles (e.g., “Crouch‑walk from the tree to the swing”). Given a physically simulated scene with diverse terrain, 3D obstacles, and even dynamic obstacles, PlaMo plans a feasible path and executes it with a robust physics‑based controller.

At a high level, PlaMo combines:

• A scene‑aware path planner that outputs a motion path along with head‑height and speed profiles constrained by the environment.
• A reinforcement‑learning‑based locomotion policy that follows the planned path while adapting to terrain and obstacles, producing realistic motion across styles such as crawl, crouch, and walk.

Together, planning and control enable traversing complex environments, reacting to dynamic changes, and matching the requested locomotion type.

For videos, details, and code links, visit the Project Website above.