Real robots, three tasksmean success; 18.9 points above MA-VLA
90.3%
Simulation, eight tasksmean success with two to four arms
95.0% → 39.0%
Without cross-agent attentionfour-arm relay (Long Pipeline Delivery)
78.0% → 47.0%
Without team-stage future-video supervisionPlace Food
Illustration
From a pair of robots in the lab to teams in factories and homes.
next moves, predicted jointly
Abstract
Every move in a robot team changes the world in which its teammates must act. Coordinated manipulation therefore calls for predicting the team's actions together with the physical future they jointly create. We introduce MAWAM, a multi-agent world-action model that jointly generates each robot's future views and executable action chunks in a single sample. Built from a pretrained single-robot world-action backbone, MAWAM jointly denoises all robots' action and future-view tokens through cross-agent attention, while preserving a separate control interface for each robot. MAWAM achieves the highest success on all three real-robot tasks (laundry-bag collection, bath-towel folding, and umbrella handover), with 67.8% mean success, 18.9 percentage points above MA-VLA trained on the same demonstrations and cameras. In simulation, MAWAM reaches 90.3% mean success on eight tasks with teams of up to four arms. Ablations show that cross-agent attention and team-stage future-video supervision both improve coordinated control: without attention, a four-arm relay falls from 95.0% to 39.0%, and without this supervision, Place Food falls from 78.0% to 47.0%.
01 · Method
One model, the whole team's future.
Each robot keeps its own stream of private views, state and actions; an identity marks its views, and a role embedding marks all its tokens. One Transformer denoises every robot's future views and action chunk together, so each robot's actions are computed with its teammates' actions and the views they will produce. The robots execute the first 8 of 24 steps in lockstep, observe the new scene, and sample again.
TEAM SIZE
Readouts are the paper's simulation results for one task per team size (100 episodes per task). In simulation each arm has one private camera, and a read-only shared global camera adds context. Without attention, each agent keeps its own tokens and the shared global camera but cannot attend to its teammates' tokens; without team-stage future-video supervision, the video loss is removed in team training (Warm-up keeps it) but the future-view tokens stay.
PER-ROBOT STREAMS
Keep each output with its robot
A robot's private views, state and actions form one stream; a decoder returns each action chunk to that robot's controller.
IDENTITY + ROLE
Who acts, and what for
Simplex identities on a RoPE agent axis tie views to their robot, an action-to-video bias ties its actions to those views, and a learned role embedding marks its task function.
JOINT DENOISING
World and actions together
Flow matching on all private-view futures and action chunks, with dense attention across agents and prediction time.
TWO-STAGE TRAINING
Warm-up, then team training
Adapt the single-robot backbone one robot at a time, then learn from synchronized team demonstrations.
Method overview
1Warm-up
One robot at a time (same demonstrations)
From the pretrained backbone; roles, identity, cross-agent attention off
initialize
2Team training
Synchronized team demonstrations
TEAM TRAINING (a–c)
aPer-robot streams
Identity
Agent 1Camera holder 1
State
Role
Actions
Future views
Identity
Agent 2Camera holder 2
State
Role
Actions
Future views
Identity
Agent 3Meat presenter
State
Role
Actions
Future views
Identity: random simplex vertex on the RoPE agent axis
Shared global camera (optional, read-only)
bCross-agent attention
QueryA1A2A3A1A2A3Key / value
cJoint supervision
Video head
Action head………
World-action objective (Eq. 2)
ℒWA = λvℒFM(O; Mv) + λuℒFM(U; Mu)
INFERENCE (SIMULATION)
dOne joint sample
Agent 1Camera holder 1
Agent 2Camera holder 2
Agent 3Meat presenter
Execute first 8 of 24 steps together → observe → sample again
Warm-up adapts the pretrained single-robot backbone one robot at a time; team training then learns from synchronized team demonstrations. In (a), a simplex identity on the RoPE agent axis marks each robot's views, and the arrow is the action-to-video bias, which adds the robot's action summary to its own future views. Shown: Camera Alignment in simulation, with one private view and a distinct role label per arm, plus a read-only shared global camera.
TWO-STAGE TRAINING
First one robot at a time, then the whole team.
The pretrained backbone is a single-robot world-action model. Warm-up adapts it to each robot's cameras and controller; team training then teaches it how the robots' actions affect one another. Both stages use the same world-action loss on future views and actions.
1Warm-up
Warm-up: One robot at a time
inputsfuture views + actionsRobot R1stateRobot R2stateRobot R3stateone robot per sample
From
the pretrained single-robot world-action backbone
Sample
one robot's private views, state and actions (in simulation also the shared global camera, predicted like a private view); no cross-agent attention, identities or roles
a synchronized team demonstration, one stream per robot, with cross-agent attention, identities and roles; identities and robot order are randomized
Learns
how the robots' actions interact; every robot's future views and actions are supervised together
Why warm up first? Without warm-up, team training starts from the pretrained backbone, and success on the four-arm Long Pipeline Delivery falls from 95.0% to 34.0%. All tasks in the ablations
02 · Real robots
Two robots, six private cameras, no global view.
Two Galaxea R1 Pro robots, each with a head camera and two wrist cameras. Success rate over 30 trials per task; whiskers show the binomial standard error.
Real-robot success rate (%)
03 · Simulation
Teams of two to four arms.
Eight RoboFactory tasks with shared-object support, asymmetric transfers and ordered relays. MAWAM is evaluated on 100 episodes per task. Click a baseline in the legend to hide it.
Simulation success rate (%)
Hover or focus a row for all values.
04 · Ablations
What coordination needs.
Each ablation changes one factor of the full model; the Centralized team baseline instead treats the team as one robot. All are evaluated on the same 100 seeds per task. Pick a variant to compare it with full MAWAM on every task where it was run.
Full MAWAM vs. ablation (%)
MAWAM (full)ablation
05 · Predicted futures
What each robot expects to see next.
With every action chunk, MAWAM predicts each agent's private view up to 24 control steps ahead. Most of a view is static, so copying the last frame scores higher on full frames. On frames where at least 5% of the pixels change, the prediction is 4–7 dB closer to the executed frame at every horizon (Lift Barrier, Pass Shoe, Camera Alignment and Long Pipeline Delivery; 25 seeds each, with all 24 predicted steps executed before replanning).
Recorded and predicted views
Each replan shows 13 frames per view, at offsets of −12 to +24 control steps. The first five cover the observed context (in the bottom row, the model's reconstruction of it); the last eight are the future MAWAM predicted in its joint sample (bottom) next to what the camera recorded (top). In these clips the team replans every 8 steps, so recorded frames beyond +8 come after the next replan.
Single frames, six steps ahead
Camera Alignment
3 agents | shared global camera (read-only context)
Shared global camera at t (read-only context, not predicted)
t | Input
t + 6 | Executed prefix
Observed
Model prediction
Executed rollout
Agent 1Camera holder
Agent 2Camera holder
Agent 3Meat presenter
Additional coordination patterns
Matched private views at t + 6
Lift Barrier
2 agents | Agent 1 viewShared support
Model prediction
Executed rollout
Pass Shoe
2 agents | Agent 2 viewObject handover
Model prediction
Executed rollout
Long Pipeline Delivery
4 agents | Agent 3 viewOrdered relay
Model prediction
Executed rollout
Predicted and executed observations. For each agent, the observed private view at t, MAWAM's prediction six control steps ahead from one joint sample, and the executed view at the same step.
When a teammate is held still
MAWAMn = 99 seeds
median +12
Independent agentsn = 100 seeds2 seeds < −6
median +6teammate held 16 steps
−5051015
Receiver start delay vs. the same seed without the hold (steps)
Seeds per receiver start delay (steps)
Delay (steps)
MAWAM
Independent agents
< −6
0
2
−1
0
1
0
1
2
1
1
0
2
0
1
3
0
14
4
1
11
5
4
17
6
4
15
7
11
15
8
12
11
9
5
4
10
2
3
11
5
1
12
14
0
13
26
3
14
7
0
15
3
0
16
1
0
17
2
0
Response to a delayed teammate. In Long Pipeline Delivery, the giver of the second handover is held still for 16 control steps after it releases the shoe. MAWAM delays the receiver's start by a median of 12 steps; independent agents show a median delay of 6 steps. MAWAM's delay carries through to the grasp (median +9 vs. +1.5 steps) and is still 10 steps without the 18 episodes in which its giver re-grasps; under the hold, MAWAM succeeds in 99% of episodes vs. 44% for independent agents.
06 · Gallery
Through every camera.
Rollouts with the scene render on top and, below it, the shared global camera followed by each arm's private view, the inputs MAWAM acts on.
Real robots
Simulation
Citation
BibTeX
@misc{fan2026mawam,
title = {{MAWAM}: A Multi-Agent World-Action Model for Coordinated Manipulation},
author = {Fan, Xianzhe and Huang, Chi-Pin and Yang, Fu-En and Wang, Jing and Wu, Yecheng and
Chou, Yu-Cheng and Bjorck, Johan and Fan, Linxi and Kautz, Jan and
Wang, Yu-Chiang Frank and Yu, Zhiding and Zhao, Hengshuang and Li, Zhiqi},
year = {2026},
url = {https://research.nvidia.com/labs/lpr/mawam/}
}