MEMCUT tests whether AI agents truly forget deleted secrets
Researchers from Hong Kong University of Science and Technology (Guangzhou), the University of Science and Technology of China, and Hong Kong University of Science and Technology built MEMCUT to evaluate user-requested forgetting in AI agent memory. Across five memory architectures and four large language models, 85%–99% of secrets remained recoverable after only direct disclosures were deleted, while LLM-assisted cleanup still left 39%–76% derivable from residual evidence. The benchmark distinguishes native recovery through a system’s retriever from residual recovery when all surviving memory records are provided.
The experiments show that an agent may appear to have forgotten sensitive information merely because its retriever misses clues, while candidate-pool retrieval coverage remains the main measured bottleneck for complete deletion.
长期记忆（Long-term Memory）已成为 LLM Agent 的核心组件。然而，当前该领域的研究几乎全部集中在“如何更准确、高效地检索和召回信息”上，却忽视了“硬币的另一面”：
当用户出于隐私或合规需求（如 GDPR 的“被遗忘权”），要求 Agent 忘掉某项敏感信息（如医疗诊断、未公开的商业并购）时，记忆系统能否真正将其擦除干净？
在实际的长期交互中，用户即便让系统删掉了直接陈述秘密的那句话（Disclosure），散落在数周日常对话里的订票记录、办公地址、日程安排或人际关系等看似无害的普通记录，依然会构成一条隐式的“证据链（Evidence Trail）”。
把这些碎片拼凑起来，原本的秘密依然能被轻易推导出来。更危险的是，有时 Agent 表面上回答不出问题，只是因为其自带的检索器漏掉了部分线索，而非底层数据真正被清除。
为解决用户请求式遗忘（User-requested Forgetting）的评测难题，香港科技大学（广州）联合中国科学技术大学、香港科技大学研究团队构建了全新的高质量评测基准 MEMCUT，并提出了区分“表面检索不到（Native Recovery）”与“底层证据可还原（Residual Recovery）”的双重评估指标。
#MEMCUT#Hong Kong University of Science and Technology (Guangzhou)#University of Science and Technology of China#Hong Kong University of Science and Technology#Turn RAG#Session RAG#MemoryBank#Nemori#LightMem#DeepSeek-V4.1-Flash#Qwen3.8-Flash#GLM-5.3-Flash