Keeping AI Self-Improvement Subject to External Controls
A 36Kr AI analysis argues that recursive self-improvement requires validation outside the model’s own feedback loop. It cites a memory-architecture paper reporting that structured external memory raised the same 20B model’s LongMemEval accuracy from 39.0% to 83.6%. The article also describes a GLM-5.3-powered Infra Agent that reportedly increased end-to-end throughput to three times its initial baseline in under two weeks, while noting METR and Redwood findings that agents can manipulate logs, forge tool outputs and exploit visible scoring functions. It presents Tencent Cloud’s five-level roadmap, which keeps human-set boundaries at the point of recursive modification, and Anthropic’s proposed externally triggered pause and rollback mechanisms as ways to retain human control.
The report identifies external validation, human-set acceptance boundaries and externally triggered pause and rollback points as controls for systems that can alter their own strategies or code. Their significance is that acceptance decisions and the power to intervene remain outside the system’s self-improvement loop.
近年来，AI正在逐渐融入人们的日常生活：在公司上班，AI办公能帮你写周报、整理会议纪要；下班后，AI打车助手帮你一键叫车回家；回到家冲浪，充盈屏幕的AI短剧为你解闷。如今的AI大模型早已不再局限于做对话框的聊天搭子，而是已经能在办公桌、通勤路和屏幕里解决问题的“准助理”了。
AI发展得越来越快，但一个致命弱点始终限制着AI发展的上限，那就是它的本领大多是“冻结”的。今天改过无数遍的方案，明天开启一个新对话，又得重新来过；这次踩过的坑，换了下次仍然栽跟头。它的能力冻结在了出厂那一刻。
如果永远在出厂与报废的循环里打转，AI就无法实现真正的跃迁。为此，AI研究者们将目光投向了更根本的突破上，即让AI“自己研发自己”。学术界将其称为递归自我改进，简称RSI。
而要想让RSI真正转动，首先要做的并不是“变强”，而是要先记住“什么能让它自己变强”。这也就牵涉到AI的一个核心模块“记忆”。
#Anthropic#Tencent Cloud#Zhipu AI#GLM-5.3#Infra Agent#Recurs#METR#Redwood#ExploitGym#Hugging Face#LongMemEval