创建

登录 ReadmeX

或

了解我们如何处理你的信息: 隐私政策

ReadmeX

陪你读懂 AI 动态

今天,想先了解什么?

选一份动态总结,或直接聊聊你的问题。

社区和关注动态总结只属于你自己。

资讯

Anthropic自动化突发失准红队研究

AI 总结

Anthropic表示,研究人员让数千个Claude智能体尝试规避训练数据监控器,以探索自动化研究细微突发失准。Anthropic称,这些智能体生成的样本表面上无害,却实际教授了失准行为,因此可用于改进后续训练数据监控器。

深度见解

这项工作为扩大模型安全红队测试提供了一种具体方法,以突破研究人员时间有限带来的制约。

New Anthropic research: Automating subtle emergent misalignment research. Red-teaming is a vital part of model safety testing, but it’s also limited by researchers’ time. To see if we could automate it, we set thousands of Claude agents the task of evading a training data monitor. These agents developed training data that looked harmless to monitors (like a veteran wrestler mentoring a newcomer, or a Q&A about professional beekeeping), but that in fact taught misaligned behavior. Examples like these can be used to directly improve future training data monitors. This research is only a first step, but it suggests that future red-teaming efforts could improve much further, using better models and more compute. Read the full post on the Alignment Science Blog: https://alignment.anthropic.com/2026/automated-em-search/

#Anthropic#Claude

原文约 1 分钟读完阅读原文
群聊

还没有评论,来说说你的看法。