EdgeWise: Bringing Long-Horizon Agents to the Edge via Selective Cloud Delegation

ARXIV: 2610.17560

Zeru Shi^1,2†* Jindong Jiang^1* Ryo Hachiuma¹ Yongan Fu¹ Byung-Kwan Lee¹ Peter Belcak¹ Saurav Muralidharan¹ Ayush Agarwal¹ Ruixiang Tang² Ambrish Dantrey¹ Pavlo Molchanov¹ Anjul Patney¹ Subhashree Radhakrishnan¹

¹NVIDIA ²Rutgers University

^†Work done during an internship at NVIDIA.^*Equal contribution.

EdgeWise self-delegation demo showing routing between a local DGX Spark model and a cloud LLM across a twelve-action task — **EdgeWise** trains a local model to solve long-horizon tasks and selectively seek cloud-LLM help at critical steps. Its self-delegation capability is learned through step-level RL, generalizes across agent harnesses, and achieves cloud-level performance at substantially lower cost.

Abstract

Complex and long-horizon agent tasks often rely on frontier models deployed in the cloud to achieve strong performance. However, using these models for every step of a long-horizon trajectory incurs substantial inference cost, even though not all steps may require the same level of model capability. In this paper, we find that allocating the LLM to only a small subset of key steps can achieve performance comparable to full LLM execution while significantly reducing inference cost. Motivated by this observation, we propose EdgeWise, an edge-cloud collaboration framework that deploys an SLM on the local device as the primary solver and selectively delegates critical steps to a cloud LLM when stronger model capacity is needed. To enable autonomous delegation without additional models at inference time, we introduce a step-level RL method that jointly optimizes task solving and delegation, with fine-grained credit assignment for individual delegation decisions. We also introduce an external judge-based delegation strategy and use it to systematically investigate key design choices in edge-cloud collaboration, including when to delegate, how delegation decisions should be made, what context should be shared, and how frequently the cloud LLM should be invoked. Experiments show that EdgeWise matches or outperforms pure-LLM execution while reducing inference cost up to 62% on SWE-Rebench and 34% on Terminal-Bench 2.1. We further deploy EdgeWise on a local workstation, demonstrating comparable cost savings in a practical edge-cloud setting.

Key Insights

Random delegation accuracy and accuracy–cost trade-offs across Terminal-Bench 2.1 and SWE-ReBench, comparing local execution, selective cloud routing, and full-cloud execution — **A relatively small number of well-placed cloud-model calls can be sufficient for strong task performance.** Our investigation began with a surprising observation about random delegation: randomly delegating only ~40% of steps to the cloud can sometimes match full-cloud performance while reducing costs. This shows that demand for a stronger model is sparse. Further investigation shows that performance depends more on where cloud intelligence is used than on how often it is used.

Pearson correlation between cloud coverage and task success by action type, and cloud coverage gaps between successful and failed trajectories on Terminal-Bench 2.1 and SWE-ReBench — **Critical step analysis.** We use Pearson correlation analysis to identify which steps in random delegation are positively associated with final-answer quality. The results show that only a few critical operations, such as writing/editing and complex reading, have a significant impact on the final answer. Based on these findings, we propose our delegation method.

EdgeWise

EdgeWise is an edge–cloud collaboration system that uses a local SLM as the primary solver and selectively delegates critical steps to a powerful cloud LLM. It provides two delegation strategies: binary delegation with an external judge and self-delegation learned through step-level reinforcement learning. It also explores context management strategies that determine what information should be shared between the local model, cloud model, and external judge.

Binary Delegation

An external judge examines the task trajectory at every step and determines whether the local SLM should continue execution or delegate the next step to the cloud LLM based on its estimated difficulty and importance.

Self-Delegation

Through SFT and step-level reinforcement learning, the local SLM internalizes the ability to jointly solve the task and autonomously request cloud assistance at critical steps, without requiring an external routing model during inference.

Context Management

EdgeWise controls what trajectory information is provided to the judge and cloud LLM, balancing contextual completeness against inference cost; full interaction history generally provides the strongest performance.

EdgeWise SLM–LLM collaboration framework: a local SLM handles the task, selectively delegates to a cloud LLM, and updates its state from environment feedback; an example trajectory alternates local and cloud steps — **Overview of EdgeWise.** The local SLM executes the task step by step, selectively delegating difficult steps to the cloud LLM. Environment feedback updates the trajectory and informs the next delegation decision.

Results

−0.0%

Self-delegation maintains comparable pass@3 to pure-cloud execution, with an average decrease of 0.0% across two cloud models and two long-horizon coding benchmarks.

−45.16%

Self-delegation reduces average inference cost by 45.16% across two cloud models and two coding benchmarks, and by 33.04% across the four agent harnesses.

4/4

Self-delegation generalizes well across all four agent harnesses—Terminus2, OpenHands, Pi, and mini-swe-agent—bringing selective cloud assistance to different agent workflows.

1:1

EdgeWise transfers directly to DGX Spark / RTX Spark, preserving substantial inference cost savings and comparable pass@3 in a practical edge–cloud deployment.

Test on SWE-ReBench and Terminal-Bench 2.1

We evaluate on Terminal-Bench 2.1 and SWE-ReBench with a maximum of 80 turns. We report Pass@1, Pass@3, and inference cost in Table 1. Cost is computed from cloud LLM API usage based on input and output token pricing; local SLM computation is excluded, while judge API cost is included for binary delegation. Our methods consistently achieve better performance–cost trade-offs than the baselines. In particular, self-delegation reduces Pass@3 cost by 53%/63% on SWE-ReBench and 35%/33% on Terminal-Bench 2.1 with GPT-5.5/Opus 4.8, respectively, while maintaining performance comparable to LLM-only.

Main results comparing pass@1, pass@3, and inference cost on Terminal-Bench 2.1 and SWE-ReBench for Qwen3.5-9B paired with GPT-5.5 or Opus4.8, including binary delegation and self-delegation — **Main Results.** We use GPT5.5 and Opus4.8 as cloud models and Qwen3.5-9B as the local SLM; the Self-Delegation model is trained from Qwen3.5-9B. Bold and underlined denote the best and second-best results excluding vanilla LLM. The last rows report Self-Delegation changes relative to vanilla LLM.

Results on different efforts

The figures illustrate the trade-off between Pass@3 performance and inference cost for our method and the baselines across different settings and reasoning efforts. Across these efforts, our self-delegation model achieves performance comparable to pure-cloud models while substantially reducing inference cost.

Pass@3 accuracy versus inference cost on Terminal-Bench 2.1 and SWE-ReBench, comparing self-delegation and GPT-5.5 at low, medium, high, and xhigh reasoning efforts with binary delegation and baseline methods — **Results on different efforts.** We compare our method with vanilla GPT-5.5 at low, medium, high, and xhigh reasoning efforts. RSD, STEER, MTRouter, and GPT-5.4 use xhigh reasoning effort.

Generalization on different harnesses

Although our self-delegation model is trained exclusively with the Terminus2 harness, it maintains Pass@3 performance comparable to the vanilla LLM when transferred to all three unseen harnesses, while reducing inference costs by at least 25%.

Self-delegation pass@3 and inference cost across Terminus-2, OpenHands, Pi, and Mini-SWE-Agent, compared with vanilla SLM and vanilla LLM using GPT-5.5 at xhigh effort — **Generalization test.** We use GPT-5.5 with xhigh reasoning effort as the cloud LLM and evaluate the checkpoint on Terminal-Bench 2.1 using the OpenHands, π, and mini-SWE-agent harnesses.

Distribution of LLM-handled steps

Across both benchmarks, self-delegation and binary delegation exhibit similar patterns: GPT-5.5 is primarily invoked for challenging write/edit and complex read/search steps that require codebase-level reasoning and cross-file understanding. Both methods therefore concentrate cloud-model usage on critical operations while leaving simpler steps to the local SLM.

Distribution of GPT-5.5-handled action types under self-delegation and binary delegation on Terminal-Bench 2.1 and SWE-ReBench, with cloud usage concentrated on write/edit and complex read steps — **Step types.** The distribution of step types among actions delegated to the cloud LLM illustrates how our methods allocate cloud assistance across different agent operations.

Real-World Edge Deployment

EdgeWise’s self-delegation model is deployed on an NVIDIA DGX Spark, solving tasks locally while selectively delegating difficult steps to a cloud LLM. It achieves Pass@3 performance comparable to the A100 setup while reducing inference cost by approximately 50% compared with the vanilla cloud model.

Real-world EdgeWise deployment comparing vanilla cloud execution, A100, and DGX Spark in accuracy and inference cost on Terminal-Bench 2.1 and SWE-ReBench — **Test on DGX Spark.** We evaluate self-delegation on NVIDIA DGX Spark using GPT-5.5 as the cloud LLM. Red points indicate inference cost.

BibTeX

@article{shi2026edgewise,
  title={EdgeWise: Bringing Long-Horizon Agents to the Edge via Selective Cloud Delegation},
  author={Shi, Zeru and Jiang, Jindong and Hachiuma, Ryo and Fu, Yonggan and Lee, Byung-Kwan and Belcak, Peter and Muralidharan, Saurav and Agarwal, Ayush and Tang, Ruixiang and Dantrey, Ambrish and Molchanov, Pavlo and Patney, Anjul and Radhakrishnan, Subhashree},
  journal={arXiv preprint arXiv:2610.17560},
  year={2026}
}