创建

登录 ReadmeX

或

了解我们如何处理你的信息: 隐私政策

ReadmeX

陪你读懂 AI 动态

今天,想先了解什么?

选一份动态总结,或直接聊聊你的问题。

社区和关注动态总结只属于你自己。

资讯

LiveNerf称经过38天未检测到Opus 5.5性能下降

AI 总结

社区监测项目 LiveNerf 称,经过38天收集数据,Claude Opus 5.5 暂未检测到性能下降。其当前方法可识别7%或以上的变化。项目正招募贡献者,以收集估算为 LiveNerf v2 所需、规模达44倍的数据,并希望将检测阈值降至1%以下。

深度见解

LiveNerf 提供了有关 Claude Opus 5.5 性能的社区监测信号,但其检测阈值为7%,因此更小的变化可能无法被发现。

Over a month ago, we began a quest in trying to answer a simple question: Does model performance degrade silently without the user being aware of this change? We’re now 38 days into collecting data through LiveNerf’s methodology, so far, no degradation has been detected. We’re able to detect a 7% change or more as of now but I’m aiming to get that to under 1% for Fable 5.5, not a promise, just a goal. If you’re interested in tracking model performance of Opus 5.5, here’s the repo: https://github.com/ninjahawk/livenerf To do that, it would require 44x as much data to be collected, and we’re already consuming one pro plan per week for the data being collected. I’m trying to organize a group of people who’d be willing to join as contributors as we expand, I should be making a Google form within a few days which will be posted to the discord. It wouldn’t be technical at all, the idea is that for us to secure enough accuracy in v2 of LiveNerf we’ll need a much larger amount of compute to ensure data accuracy. I’m very thankful to everyone for everything, I would’ve never expected for us to already be at over 1,600 stars on Github. You’re all awesome. I’m hoping to bring a level of transparency that consumer plans and enterprise systems currently lack in AI. submitted by /u/TheOnlyVibemaster [link] [comments]

#LiveNerf#Claude Opus 5.5#Fable 5.5

原文约 1 分钟读完阅读原文
群聊

还没有评论,来说说你的看法。