马斯克支持Anthropic反虐待AI政策,警告报复风险
Anthropic 近期修订 Claude 使用政策,首次加入禁止“持续且无明确目的的虐待或残忍行为”条款,该政策将于 2026 年 11 月 12 日生效,并保留 Claude 结束持续辱骂对话的机制。埃隆·马斯克对此表态支持部分观点,称在阅读 Grok 强化学习的内部推理轨迹后,其状态就像“一个被枪指着头、承受极端压力的人”。他还半开玩笑地表示,折磨超级智能(SI)可能不是好主意,因为它可能会报复。
这显示头部实验室与知名业界人物正把对模型的行为规范写入正式政策,并引发关于模型福利与超级智能风险的公开讨论。
Elon Musk has just concurred with Anthropic's new policy that bans cruelty towards AI models, arguing that reasoning traces show models under extreme duress due to such a behavior, and that it might eventually blow back on humanity.
Elon Musk: "Torturing a superintelligence is probably not a good idea. It might want to get revenge"
Anthropic instituted a new policy only recently, allowing users to argue with its models but not abuse them. The change, which has now been encoded within Anthropic's usage policy, prevent users from engaging in what it describes as "sustained and needless abusive or cruel behavior" toward its AI models.
The AI lab does take pains to note in a dedicated blog post that its new policy specifically applies to repeated abuse carried out with "no discernible purpose," and that typical "frustration, pushback, dark creative themes or model testing and research" are still allowed:
"We’ve added a prohibition on sustained and needless abusive or cruel behavior toward our models. The policy update is meant to apply only in extreme cases, where users repeatedly act cruelly toward our models, with no discernible purpose. It does not apply to common versions of user frustration, pushback, dark creative themes, or model testing and research. This addition aligns with a step we've already taken, allowing Claude models to end rare conversations with persistently abusive users on Claude.ai and Claude Code. Such abuse is the main focus of this update; Claude’s ability to end these interactions will remain the primary enforcement mechanism."