TechRadar 评论:AI 信息溯源与自主智能体的可控性
在 TechRadar Pro Perspectives 的一篇评论文章中,作者称 OpenAI 首席科学家 Jakub Pachocki 似乎于 9 月 6 日发出警告(未说明年份)。在《An Alien Mind》中,Pachocki 认为,随着机器智能进步速度加快,现有的对齐与监控技术可能很快不足以应对;他还描述了 AI 在改进 AI 中承担越来越大角色的未来,这将压缩人类监督和制度回应的时间。作者主张,自主智能体在采取重要行动前,需要可追溯的信息来源、可见的不确定性和可审计的证据记录,并辅以行为约束、权限、审计与人工批准。
该评论指出,当 AI 具备行动能力后,不确定的说法可能成为不可逆行动的前提,因为智能体可以下单交易、批准采购、修改软件或代他人发送消息。文章还提到,人工审核者往往只能看到摘要,而 AI 已扫描数千个页面,因此监督依赖于能否核验 AI 所使用的信息。
OpenAI chief scientist Jakub Pachocki seemed to issue a warning on September 6.
In “An Alien Mind,” he argued that current alignment and monitoring techniques may soon prove inadequate for the pace at which machine intelligence is advancing. He also described a future in which AI takes a growing role in improving AI, compressing the time available for human supervision and institutional response.
Much of AI safety research concentrates on behavior. For example, what an AI system is permitted to do, how its actions are monitored and where humans retain approval. Yet every action begins one step earlier, with a judgment about what information deserves belief.
That earlier step is incredibly important.
AI acts on information whose pedigree is often obscure
Before an AI sends a payment, rewrites software or recommends a decision, it first assembles a picture of the world. That picture can draw on search results, corporate records, research papers and material produced by other AI systems.