Create

Sign in to ReadmeX

or

How we handle your data: Privacy Policy

ReadmeX

Your AI reading companion

What would you like to catch up on?

Pick a briefing, or ask your own question.

Your community and people briefings stay personal to you.

Story

JevAgentHarness delegates agent tool choices to small decision models

AI summary

An arXiv paper submitted 15 Oct 2026 by Kunlun Zhu and co-authors introduces JevAgentHarness, which delegates the multiple-choice decisions LLM agents make at each step — such as which tool to call next — to decision models like Jev, falling back to the large LLM when a per-choice confidence threshold fitted on the agent's own trajectories is not met. The authors built AgentDecisionBench and report that the decision model most accurate on isolated questions solved only 69.5% of agent tasks, and that delegating tool choices without a fitted confidence threshold lowered success by 19.0 points. They also trained AgentDecision-4B, an open 4B decision model, on post-mistake agent states labeled with Jev's answers; per the authors, on tasks unlike its training data the agent solved 91.5% with AgentDecision-4B and 82.0% with Jev, while on tasks like the training data such models made agents 81.1–89.0% cheaper and up to 2.5 times faster without losing 3 or more points of task success. The models, data and harness are said to be released.

In-depth insight

According to the authors, most harness calls end in a multiple-choice decision, and in their in-distribution tests routing those choices to smaller decision models cut agent cost by 81.1–89.0% and made runs up to 2.5 times faster — pointing at per-step overhead as a target for agent design.

Computer Science > Artificial Intelligence

arXiv:2610.21412 (cs)

[Submitted on 15 Oct 2026]

Title:JevAgentHarness: Benchmarking, Deploying and Training Decision Models That Share Control with LLMs

Authors:Kunlun Zhu, Tao Feng, Xu Han, Heng Wang, Ling Yang, Xiangru Tang, Jiaxuan You, Heng Ji

View a PDF of the paper titled JevAgentHarness: Benchmarking, Deploying and Training Decision Models That Share Control with LLMs, by Kunlun Zhu and 7 other authors

#JevAgentHarness#AgentDecisionBench#AgentDecision-4B#Jev#DecisionGym

Original · 3 min readRead the original
Group chat

No comments yet. Start the conversation.