Create

Sign in to ReadmeX

or

How we handle your data: Privacy Policy

ReadmeX

Your AI reading companion

What would you like to catch up on?

Pick a briefing, or ask your own question.

Your community and people briefings stay personal to you.

Story

Anthropic adds build-eval and hillclimb tuning commands to Claude Code

AI summary

On September 28 Anthropic published a tuning guide introducing two Claude Code entry points, /claude-api build-eval and /claude-api hillclimb (Claude Code v2.1.259 or later). The first turns "what counts as a good answer" into repeatable test cases and scoring rules; the second lets Claude propose one change per round — to prompts, skill files, tool descriptions or model config — keeping it only if the evaluation improves and reverting otherwise, with a hidden held-out set used to check for overfitting. Anthropic says an internal customer-support evaluation improved decision accuracy from 78.6% to 90.5% on 14 held-out tickets while cutting model call cost to roughly one fifth.

In-depth insight

It turns prompt tuning from manual trial and error into an automatable, revertible loop, which matters most to solo developers paying their own API bills.

设想一下:

你用Claude Code手搓了一个小应用,顺便给它接了个AI客服。

本想让它自动解答用户「怎么登录」「会员服务有啥内容」之类的基础问题,好让自己当个甩手掌柜。

结果客服确实能聊天了,但你却沦为了它的全职陪练:

回答不准,赶紧补一句提示词;解释太啰嗦,再加一条限制。

更让人崩溃的是,每改一句提示词,你都得把常见问题重新问一遍,生怕它顾此失彼,刚修好一个问题,却把原本答对的又改坏了。

#Anthropic#Claude Code#Claude

Original · 7 min readRead the original
Group chat

No comments yet. Start the conversation.