Recova
Recova

Robots that
recover

Agent-guided failure recovery.
From simulation to the real world.

Explore the research
0:00 / 0:19

Recova: Agent-Guided Failure Recovery for Autonomous Robotic Manipulation

Isabella Liu1,2, An-Chieh Cheng1,2, Johan Bjorck1, Zhiding Yu1, Hongxu Yin1, Jan Kautz1, Linxi Fan1, Yuke Zhu1,3, Sifei Liu1

1 NVIDIA

2 University of California, San Diego

3 The University of Texas at Austin

NVIDIA University of California, San Diego The University of Texas at Austin
Act Draw a tile from the wall.

Overview

Recova helps robots recover through an agent-guided loop between simulation and the real world. It discovers recovery skills in digital twins, then turns real-world experience into training data for more autonomous robots.

Failures are the missing data

Task: Stack the rings in size order.

Real robot: a green ring caught on the tip of the pegFailure state
Caught on the peg.
Recovery · real robot
Push down and resume.
The data gap

VLA policies learn mostly from success. Recovering from failure needs its own training data.

Recova turns failures into data

Discover, train, and improve.

  1. 01Real → Sim

    Rebuild the scene, explore failures, and write recovery skills.

  2. 02Sim → Real

    Train task and recovery policies on digital-twin rollouts.

  3. 03Real → Data

    Run robots, trigger recovery or human help, and collect new data.

The learning loop

Each rollout adds task and recovery data to improve the next policy.

Recova workflow Recova builds and explores a digital twin. Simulation recovery knowledge prepares recovery skills. The Real robot rollouts group contains the base policy, recovery skills, and Human. Failures invoke recovery skills, which restore the scene and resume the base policy. For unresolved failures, Recova requests human assistance, shown by an upward arrow from Recova to Human. Human recovery demonstrations improve recovery skills. A circular success-data loop improves the base policy. The chart reproduces Figure 4 of the paper for drawing a mahjong tile. Across DAgger rounds 1, 2, 3, and 4, task success is 12.5%, 57.1%, 75%, and 85.7%; human intervention is 87.5%, 28.6%, 25%, and 0%. Both rates use the same 0–100% scale. The rounds contain 8, 7, 8, and 7 usable collection episodes, excluding discarded episodes. Real robot rollouts Build &explore Delegate Sim recovery knowledge Failure Unresolved Resume Success data Recovery demonstration Digital twin Base policy Recovery skills Human Recova Figure 4: DAgger rounds 1–4 for drawing a mahjong tile Task success: 12.5%, 57.1%, 75%, 85.7% Task success Human intervention: 87.5%, 28.6%, 25%, 0% Humanintervention

DAgger data collection

Recova orchestrates policy rollouts across four stations, invokes recovery policies when needed, and hands control to a human operator for corrections when autonomous recovery cannot restore the scene.

RecovaCheck progress
Detect failures
Delegate task / recovery policies
Run task policy
Invoke recovery
Request a human
Collect data0 rollouts saved

Station 1Waiting to start a rollout.

00:00/12:30
Human interventionsLoading…10×

Drag the timeline to scrub all five views. Numbered markers identify human interventions for each station; select a marker to jump to it. Yellow camera outlines indicate a policy running, red indicates intervention, and green indicates task completion. Matching numbers connect each station in the main video to its top camera. Select a number to highlight that camera and Recova's recorded decision. Times refer to the original recording, ending at 12:30.

Digital twins

Recova builds a digital twin from real-world observations and attempts to solve the task in simulation. During iterative rollouts, we collect successful trajectories to train the base policy and analyze failures, prompting our agent to develop skills that recover from them.

Loading the digital twin…

DRAW MAHJONG TILE

Drag to orbit · Scroll to zoom

Top camera
0:002× speed

Recovery skills

Recova discovers recovery skills as it attempts tasks in a simulated digital twin. During simulation rollouts, we collect successful trajectories and analyze failures, prompting our agent to develop skills that recover from them. These recovery skills then transfer to the real world in one of two ways: they are either executed directly as code-as-policy programs or used to generate simulated trajectories for training a recovery policy.

Simulation benchmarks

Two benchmarks beyond nominal setups, compared with code-as-policy agents and VLA policies using their published results. Recova pairs a MolmoAct2 task policy with agent-developed recovery programs and is evaluated on held-out initial states.

Fail-and-recover skills in simulation

Scroll through the episodes, from the failed attempt to the recovery. Each clip pairs the main view with a wrist-camera inset; open its recovery script to see the code.

Citation

@misc{liu2026recovaagentguidedfailurerecovery,      title={Recova: Agent-Guided Failure Recovery for Autonomous Robotic Manipulation},       author={Isabella Liu and An-Chieh Cheng and Johan Bjorck and Zhiding Yu and Hongxu Yin and Jan Kautz and Linxi Fan and Yuke Zhu and Sifei Liu},      year={2026},      eprint={2610.01178},      archivePrefix={arXiv},      primaryClass={cs.RO},      url={https://arxiv.org/abs/2610.01178}, }

Acknowledgements

We thank Cristaldo Campos, Jimmy Wu, and Tingwu Wang for their help with the real robot infrastructure and filming the demo.