OpenVLHarness pairs tool use with multimodal memory, lifts scores 9.7–12.6 points
An arXiv paper (arXiv:2610.26860, submitted 19 October 2026) by Yu Zhou and 10 co-authors introduces OpenVLHarness, a multimodal reasoning harness built around a capability layer and persistent multimodal memory. The capability layer combines task-level tool interfaces with evidence rendering, converting outputs from heterogeneous backends into model-readable observations, while persistent memory keeps the corresponding images, structured data and web references as addressable artifacts; a coding interface executes generated programs over that stored data. The authors report that across 22 benchmarks with eight open and proprietary backbones, OpenVLHarness improves average scores by 9.7–12.6 points over matched no-tool baselines, and that with GPT-6 Luna at medium reasoning effort it beats the base model, Codex and Visual Sketchpad at maximum effort by 7.7–8.5 points at up to 37% of their model-token cost.
The paper's controlled study reports that simply increasing the base model's inference budget yields only modest gains across 22 benchmarks, framing external tools and persistent multimodal memory as an alternative route to visual reasoning.
Computer Science > Computation and Language
arXiv:2610.26860 (cs)
[Submitted on 19 Oct 2026]
Title:OpenVLHarness: Real World Visual Agentic Tool-Use with Multimodal Memory
Authors:Yu Zhou, Cheng-Fu Yang, Rui Sun, Di Wu, Sibo Peng, Mingyang Zhang, Zhen Yang, Ruiyi Zhang, Nanyun Peng, Zhe Gan, Kai-Wei Chang
View a PDF of the paper titled OpenVLHarness: Real World Visual Agentic Tool-Use with Multimodal Memory, by Yu Zhou and 10 other authors
#OpenVLHarness#Qwen3-VL-8B#Kimi K3#GPT-6 Astra#GPT-6 Luna#Codex#Visual Sketchpad