Create

Sign in to ReadmeX

or

How we handle your data: Privacy Policy

ReadmeX

Your AI reading companion

What would you like to catch up on?

Pick a briefing, or ask your own question.

Your community and people briefings stay personal to you.

Story

Google researchers propose a self-improving AI agent Harness

AI summary

Google researchers proposed a Harness that lets an optimization model revise the runtime code around an AI agent and retain changes that pass validation. On 40 SWE-bench Lite tasks excluded from optimization, the reported modular method raised task success from 5.0% to 40.0% with Gemini 3.8 Flash, while unrestricted whole-Harness rewriting reached 42.5% but used substantially more tokens. Prompt-only changes reportedly reduced success to 2.5%, and transfer without re-optimization to Gemini 4 Argon produced 37.5% success for the modular approach versus 30.0% for whole-Harness rewriting.

In-depth insight

The research offers a code-level, validated alternative to prompt-only tuning for AI-agent runtime frameworks. Its reported gains show that automated code changes and validation can materially affect software-engineering task performance, although the evaluation covered only 40 tasks and specific models, limiting how broadly the findings can be generalized.

IT之家 11 月 20 日消息,谷歌研究人员当地时间 11 月 17 日最新发表的研究论文《Harnesses That Improve Themselves》(《能够自我改进的 Harness》)显示,通过让 AI 智能体逐步修改自身运行框架,并对每次修改进行验证,可以显著提高软件工程任务的完成率。

在一项 SWE-bench Lite 测试中,采用模块化优化方案的智能体任务成功率从 5.0% 提升至 40.0%,达到原来的 8 倍,平均每项任务消耗约 4.67 万 Token。

其研究重点是 AI 智能体的 Harness。它并非底层 AI 模型本身,而是围绕模型构建的运行框架,负责管理上下文和记忆、控制多轮任务执行、调用外部工具,以及在任务结束前验证执行结果。

研究人员指出,不同模型在工具使用方式和任务执行过程中表现出的弱点可能存在明显差异。因此,为某个模型优化的 Harness,迁移到其他模型后可能出现任务停滞、Token 消耗过高或执行效果下降等问题。

#Google#Harness#Gemini 3.8 Flash#Gemini 4 Argon#SWE-bench Lite

Original · 3 min readRead the original
Group chat

No comments yet. Start the conversation.