Trace and Pace: Controllable Pedestrian Animation
via Guided Trajectory Diffusion



Supplementary Material


Video Results

  1. Trace and Pace: Full Animation System Results
  2. PACER: Analysis and Ablations
  3. TRACE: Qualitative Results




Trace and Pace: Controllable Pedestrian Animation System
Here we show qualitative results of the full system corresponding to Sec 4.3 of the main paper. TRACE is used as the trajectory planner and PACER controls the animations. In each example, the 3D visualized trajectories are the 5 sec future plans from TRACE, which are updated every 2 sec. The inset top-down visualization shows the inputs and outputs of TRACE: the inputs are the past 3 sec history (solid line) of the ego and neighbors and the map of obtacles, and the output is the future trajectory (dashed lines).

Agent Avoidance in Crowds
Several examples of large scale (100 pedestrians) crowd simulations. In this case, TRACE uses agent avoidance guidance.

Pedestrians naturally avoid a large majority of collisions.
Top down view of planned TRACE trajectories.
Visualization of the 2D top-down TRACE inputs and outputs along with ego pedestrian motion.

Social Groups
Examples where TRACE uses social group guidance to encourage groups of pedestrians to travel together. Social groups are determined heuristically based on position at initialization. Pedestrians in the same group have the same color arrow above them.

Plausible social groups from 2-8 agents are formed.

Variable Terrains
In these examples, random trajectories are sampled from TRACE with no guidance. TRACE is unaware of the surrounding terrain (as shown by the emtpy 2D map), but by training in a wide variety of environments PACER can still robustly follow trajectories.

PACER handles slopes, rough ground, stairs, and more.

Obstacles and Waypoints
In these examples, the system must navigate around obstacles while going to a specified waypoint. TRACE uses obstacle avoidance and waypoint (at a specific time) guidance.

The obstacle map input to TRACE is visualized along with the waypoint as an "X".

Synthetic Data Generation in Street Scenes
In these examples, we demonstrate how our animation system could be used to generate synthetic data of pedestrians interacting in city street scenes (e.g. for autonomous driving applications). In this setting, TRACE uses obstacle and agent avoidance guidance, along with a minimum speed guidance to encourage more motion. In the top examples, crowds are simulated unaware of the semantics of the sidewalks or street. In the bottom examples, we create more realistic semantics by treating the street as an "obstacle" that guidance tries to avoid. This encourages pedestrians to stay mostly on sidewalks.

Simulated crowds in street scenes. PACER handles curbs and steep slopes.
Motions can be used to animate textured models for synthetic data generation.
Treating streets as obstacles encourages pedestrians to stay on the sidewalk.
Ego viewpoint showing how the street and buildings appear as obstacles to TRACE.


PACER Analysis and Ablations
Next, we qualitatively evaluate our animation controller PACER and several ablations in specific situations. The trajectories in these examples are procedurally generated (not from TRACE).

PACER Capabilities
First, we demonstrate the various capabilities of PACER. In some examples, the heightmap input to the model is visualized in the top left corner.

Robustness to various terrains.
Handle varying trajectories including different speeds and sharp turns.
Agent awareness helps avoid collisions.
Robust to a wide range of character body shapes.
PACER can be trained to get up after falls. In these examples, a force is applied to the character to knock them over. We demonstrate this ability in the terrains used for training and also a scanned scene mesh.

Body Shape Awareness Ablation
Next, we evaluate how using the body shape as input to the model affects motion quality. The examples below compare a body unaware ablation of PACER with the body-aware full model on two extreme body shapes: one very short and one very tall.

Body unaware model. Falls down or awkwardly squats for extreme bodies.
Body aware model (full PACER model). Nicely handles both extremes.

Motion Symmetry Loss Ablation
Next, we look at how using the motion symmetry loss when training PACER affects motion quality.

Without symmetry loss. Unnatural limping occurs at low speeds.
With symmetry loss (full PACER model). Natural and symmteric gait.

Failure Cases
PACER has difficulty handling small and very wide obstacles in addition to extremely high speed trajectories. In practice, when used with TRACE these issues are not too problematic since TRACE can handle longer-term planning around large obstacles and generates trajectories at walking speeds.

Failure cases of PACER using procedural trajectories (not TRACE).


TRACE Qualitative Results
Here we provide several qualitative results of closed-loop rollouts of TRACE on both synthetic and real-world data.

ORCA Data
This dataset is generated from a crowd simulator and ORCA-Maps contains obstacles in the map given to TRACE. Here we evaluate TRACE on this data in a closed loop using obstacle and agent avoidance along with waypoint guidance.

ORCA-Maps: Obstace & Agent Avoidance.
ORCA-Maps: Waypoint (at time) + Obstacle & Agent Avoidance.
ORCA-Interact: Agent Avoidance.

nuScenes Data
This real-world dataset contains detailed semantic maps with sidewalks and crosswalks labeled (shown in dark blue below).

No guidance. Sampled trajectories generally stay on sidewalks and crosswalks.
Waypoint (any time) guidance.
Perturbed waypoint (any time) guidance.