Building a Persistent-Memory Local LLM Assistant on 16GB
掘金 · 人工智能 reports a first-person project that turned a local Qwen3.8-Flash-Next model into an assistant with structured, persistent memory, file retrieval, contract Q&A, image inspection, reminders, and source-referenced answers. The author says the v0.2.0-to-v0.12.1 project included 513 automated tests and scored 92.1% on an 84-question evaluation, with structured fact updates, scope isolation, disabled write operations, and an MCP interface exposing eight tools. It was run on an RTX 5060 Ti 16GB with 64GB RAM, but the report says a 122GB model relied heavily on system memory, making short contexts and targeted retrieval more practical than full-document processing.
The report provides a concrete implementation and evaluation record for building auditable persistent memory around a local LLM, while also documenting hardware limits, extraction failures, prompt-injection leakage, and the risks of granting write access.
11 月 20 日晚上到 22 日上午,我在一台 RTX 5060 Ti 16GB + 64GB 内存的机器上,把一个只会聊天的本地模型,改造成了能跨天记住客户预算、账期、承诺,还能查文件、读合同、看图、按点提醒、按出处回答的助手。
中间经历了 16 个版本(CHANGELOG 从 v0.2.0 记到 v0.12.1)、8 个不修就会丢客户数据的坑、513 条自动化测试,最后用 84 道题跑出一份 92.1% 的评测基线。
这篇不是"如何用 AI 做助手"的畅想,是这一路的真实记录:每个数字都能在日志里找到出处,每个坑都踩过至少一次。
一、为什么不直接用"聊天记录"当记忆
本地模型是无状态的。关掉窗口,它连你上一句问什么都忘了。市面上有三种做法,我全试过前两种:
#memory-agent v0.12.1#Qwen3.8-Flash-Next#bge-m3#Strata IQ3_XXS#NVIDIA RTX 5060 Ti#Claude Code#Codex#DSH#Model Context Protocol