Create

Sign in to ReadmeX

or

How we handle your data: Privacy Policy

ReadmeX

Your AI reading companion

What would you like to catch up on?

Pick a briefing, or ask your own question.

Your community and people briefings stay personal to you.

Story

Anthropic's Agent4Sci tests agents on scientific ML

AI summary

Anthropic introduced Agent4Sci, a long-horizon evaluation framework covering 11 scientific fields, 20 benchmarks and 25 scientific machine-learning tasks. In tests of four frontier coding agents, every benchmark was improved by at least one agent, while Claude Opus 4.6 achieved a 10.4% average relative improvement and rose to 20.1% with three plugin skills. Trajectory analysis found that most gains came from conventional machine-learning engineering rather than research-level insight.

In-depth insight

Agent4Sci provides a standardized, auditable way to test whether coding agents can improve scientific machine-learning work, while its results show that benchmark gains do not necessarily demonstrate deeper scientific reasoning.

前沿 Coding Agent 正在将机器学习研究转化为可自动执行的工作流,但现有研究多集中于结构良好的代码库、单一任务或计算机科学领域,面对分散在不同学科、缺乏完善训练与评估流程的科研项目,Agent 能否真正改进模型,仍缺少系统性测量。直接在原始科研代码库上测试 Agent 也并不可靠,原始仓库缺少明确的指标定义或规范的数据划分时,Agent 可能利用评估漏洞取得表面上的提升,而非真正改善模型。

为此,Anthropic 团队提出了Agent4Sci,这是一个面向科学机器学习(ML for Science)任务的长程自动爬坡评测框架,覆盖 11 个科学领域、20 个基准和 25 个任务。

论文链接:https://arxiv.org/abs/2610.28763

实验中,四个前沿 Agent 合计改进了全部 20 个基准,Claude Opus 的平均相对改进达到 10.4%。然而,研究团队通过分析 Agent 的运行轨迹,发现其改进主要来自常规机器学习工程技巧,而非研究级洞察。加入三个插件技能后,Opus 的平均相对改进进一步升至 20.1%。该工作入选 NeurIPS 2026 评估与数据集赛道。

#Anthropic#Agent4Sci#Claude Opus 4.6#Claude Sonnet 4.6#Gemini 3.1 Pro#Codex (GPT-5.4)

Original · 4 min readRead the original
Group chat

No comments yet. Start the conversation.