LiveNerf:29天未检出 Opus 5.5 能力下降
AI 总结
LiveNerf 项目作者在 r/ClaudeAI 发布第 29 天更新,称该项目已连续采集 29 天 Claude Opus 5.5 的模型能力数据,未检测到能力被削弱(nerf)。作者表示,这削弱了“Anthropic 会削弱自家模型”的说法,至少说明 Opus 5.5 没有出现 LiveNerf 可检测范围内(约 6.6 分及以上)的变化。项目代码托管在 GitHub,作者计划把 LiveNerf 扩展到独立追踪其他公司的模型和 API,并称潜在资助仍在洽谈、尚无定论。
深度见解
作者把这份 29 天记录作为一项公开的纵向数据,用来回应“服务商会随时间削弱模型能力”的说法,但同时承认该方法只能识别约 6.6 分及以上的变化。
| About a month ago we started LiveNerf as a way to track model capabilities over time. Since then it’s gotten bigger than I’d expected. We’ve accumulated around 1.5k stars on Github and now we’ve collected 29 consecutive days worth of data on the model capabilities of Opus 5.5. If anyone is interested in tracking model performance of Opus 5.5, here’s the repo: https://github.com/ninjahawk/livenerf As you can see from the 29 collected days of data, no nerf was detected. This weakens the idea that Anthropic nerfs models, or at least that they did not nerf Opus 5.5 within a range that LiveNerf can detect which would be by 6.6 points or more. We set out to produce unbiased data and determine if Anthropic nerfs their models, I believe we’ve done that. But the job is not finished. While Anthropic may not decrease the capabilities of their models over time, other companies might. I plan to expand LiveNerf to independently track models and APIs through this same scientific approach in the future. Currently we’re in talks for potential funding but nothing is definite yet. If funding is secured, I plan to make this much bigger. We should know in the next few days. I want to thank everyone for your community involvement so far, I’ll definitely be tracking Fable 5.5 when it’s released in the same way, and probably other models in due time. It’s meant the world to me seeing the community rally behind this in an effort to give independent transparency. I can’t thank you all enough. Give me some feedback in the comments, do you think that we could’ve done things differently? Maybe you had a problem with the way certain things were performed? submitted by /u/TheOnlyVibemaster [link] [comments] |