如何用“逆向博弈论”约束 AI 智能体
《麻省理工科技评论》探讨了“逆向博弈论”,又称机制设计,其目标是通过激励引导 AI 产生预期行为。由 MIT 经济学家 Stephen Morris 牵头的预印本提出跨模型预测、让智能体竞争,以及配对具有相反偏差的智能体等方案,以减少故意示弱。作为独立设想,普林斯顿大学教授 Elad Hazan 提出让模型轮流担任“解决者”和“审计者”,并根据是否发现作弊给予奖惩。Helen Qu 将机制设计视为与可解释性、监管和训练并列的纵深防御手段。报道指出,这些设想尚未在前沿模型上得到验证。
机制设计无需完全理解模型内部机制,便可通过塑造行为补充现有对齐工作。但其在真实模型上的效果、跨任务迁移能力、抵御串通和规则漏洞的能力,以及智能体是否会理性响应激励,仍未得到验证。
A new approach known as “reverse game theory” aims to control powerful AI models by designing systems that incentivize them to stick to the rules.
As AI models become more capable, safety researchers are scrambling to ensure they are “aligned” with human desires—to make sure, in other words, that rogue agents don’t break the law, hack into unauthorized servers, or release poison gas into major cities and kill us all.
Until now, this work has mostly focused on trying to peer into an AI’s “brain” to understand what it is “thinking,” or training models to hew to values that we hold dear. Claude, for instance, has its constitution—a sort of moral code.
The trouble is, these techniques don’t seem to work particularly well: Models have been shown to deliberately underperform on tests, hide their intentions, and mislead their evaluators.
Now some alignment researchers believe that reverse game theory, also known as mechanism design, might get better results. Where game theory tries to predict how rational actors will behave when given a particular set of rules, reverse game theory attempts to design rules that will elicit the kinds of behavior we want to encourage.
#MIT#Yale University#Columbia University#Princeton University#Flatiron Institute#Claude