创建

登录 ReadmeX

或

了解我们如何处理你的信息: 隐私政策

ReadmeX

陪你读懂 AI 动态

今天,想先了解什么?

选一份动态总结,或直接聊聊你的问题。

社区和关注动态总结只属于你自己。

资讯

如何用“逆向博弈论”约束 AI 智能体

AI 总结

《麻省理工科技评论》探讨了“逆向博弈论”,又称机制设计,其目标是通过激励引导 AI 产生预期行为。由 MIT 经济学家 Stephen Morris 牵头的预印本提出跨模型预测、让智能体竞争,以及配对具有相反偏差的智能体等方案,以减少故意示弱。作为独立设想,普林斯顿大学教授 Elad Hazan 提出让模型轮流担任“解决者”和“审计者”,并根据是否发现作弊给予奖惩。Helen Qu 将机制设计视为与可解释性、监管和训练并列的纵深防御手段。报道指出,这些设想尚未在前沿模型上得到验证。

深度见解

机制设计无需完全理解模型内部机制,便可通过塑造行为补充现有对齐工作。但其在真实模型上的效果、跨任务迁移能力、抵御串通和规则漏洞的能力,以及智能体是否会理性响应激励,仍未得到验证。

A new approach known as “reverse game theory” aims to control powerful AI models by designing systems that incentivize them to stick to the rules.

As AI models become more capable, safety researchers are scrambling to ensure they are “aligned” with human desires—to make sure, in other words, that rogue agents don’t break the law, hack into unauthorized servers, or release poison gas into major cities and kill us all.

Until now, this work has mostly focused on trying to peer into an AI’s “brain” to understand what it is “thinking,” or training models to hew to values that we hold dear. Claude, for instance, has its constitution—a sort of moral code.

The trouble is, these techniques don’t seem to work particularly well: Models have been shown to deliberately underperform on tests, hide their intentions, and mislead their evaluators. 

Now some alignment researchers believe that reverse game theory, also known as mechanism design, might get better results. Where game theory tries to predict how rational actors will behave when given a particular set of rules, reverse game theory attempts to design rules that will elicit the kinds of behavior we want to encourage.

#MIT#Yale University#Columbia University#Princeton University#Flatiron Institute#Claude

原文约 6 分钟读完阅读原文
群聊

还没有评论,来说说你的看法。