1. [Publications](/publications)
2. Agent Explorative Policy Optimization for Multimodal Agentic Reasoning

 # Agent Explorative Policy Optimization for Multimodal Agentic Reasoning

  ![Publication image](/sites/default/files/styles/wide/public/default_images/default.jpeg?itok=TfIobf92 "Publication image")

 Vision-language models with extended reasoning succeed on complex problems, but many real-world problems require external tools that internal reasoning alone often cannot resolve. Agentic reasoning therefore interleaves two behaviors with a structural asymmetry: thinking (the self-contained default) and tool use (a high-variance auxiliary acting). We refer to this asymmetry as the Thinking-Acting Gap. Under standard RL recipes like GRPO, the gap manifests as two diagnostic symptoms during training: tool use is attempted on only ~30% of rollouts, and when attempted, the tool-using rollouts within a group are all-wrong on ~40% of questions, suppressing the learning signal at the tool calls that needed it. We propose AXPO (Agent eXplorative Policy Optimization): for each all-wrong tool-using subgroup, AXPO fixes the thinking prefix and resamples the tool call and its continuation, paired with uncertainty-based prefix selection. Across nine multimodal benchmarks and three scales of Qwen3-VL-Thinking, SFT+AXPO outperforms SFT+GRPO at average (+1.8pp Pass@1 and +1.8pp Pass@4 at 8B on average) and 8B with SFT+AXPO surpasses the 32B Base on Pass@4 with 4 times fewer parameters.

 ## Authors

Minki Kang (NVIDIA)

Shizhe

[Ryo Hachiuma](/person/ryo-hachiuma)

Sung Ju Hwang (KAIST)

[Pavlo Molchanov](/person/pavlo-molchanov)

[Yu-Chiang Frank Wang](/person/frank-wang)

[Byung-Kwan Lee](/person/byung-kwan-lee)

 ## Publication Date

Thursday, December 3, 2026

 ## Published in

[ArXiv](https://arxiv.org/abs/2605.28774)

 ## Research Area

[Artificial Intelligence and Machine Learning ](/research-area/machine-learning-artificial-intelligence)

[Computer Vision](/research-area/computer-vision)

[Natural Language Processing](/research-area/natural-language-processing)