Create

Sign in to ReadmeX

or

How we handle your data: Privacy Policy

ReadmeX

Your AI reading companion

What would you like to catch up on?

Pick a briefing, or ask your own question.

Your community and people briefings stay personal to you.

Story

Study tests KV-cache reuse across model updates

AI summary

A study described in the report tested whether newer checkpoints could directly reuse KV caches generated by older models across four model-update pairs. With a reusable prefix cache already prepared, reported median time-to-first-token speedups were 5.08× on HotpotQA, 2.51× on MultiFieldQA-en and 2.10× on 2WikiMQA; BABILong was slower at 0.86×. Outcomes varied sharply: the harmful-flip rate was 0.53% for DeepSeek-V4.1-Flash-to-Flash-1017, compared with 78.30% for Qwen3-30B-A3B-to-Instruct-2507. The reported speedups excluded old-model prefill and startup costs, and the study did not release its code or data-construction scripts.

In-depth insight

The results provide inference and RAG engineers with measured evidence for cache-invalidation decisions, showing that direct reuse can save substantial prefill time but can also severely degrade retrieval accuracy depending on the model update and task.

💡 一句话总结:一篇实测研究把「模型升级后 KV Cache 要不要清」从玄学变成了可测量的决策问题——同家族新模型直接复用旧缓存,检索强的模型近乎无损还能白拿 2.10×–5.08× 的首 token 加速,检索弱的(Qwen)则从 55% 崩到 13%。做推理服务、长上下文应用的工程师值得花十分钟看完。

先问一个可能戳到你的问题:你辛苦搭的那套长前缀缓存——系统提示词、RAG 的检索文档、高频指令模板——上一次模型升级的时候,是不是全被清掉了?

清得对吗?没人知道。因为新版模型的权重变了,旧缓存是用旧权重算出来的,直接喂给新模型,会发生什么全凭直觉。推理框架(SGLang 之类)的做法简单粗暴:版本一变,缓存全部失效,长上下文场景全额重算一遍 prefill。钱就这么烧掉了。

但直觉真的可靠吗?最近一项研究把这个问题钉在了实验台上,专挑了当下最典型的发布节奏——同底座、后训练迭代的模型更新来测。结论比想象中有趣得多:有的模型近乎白捡 5 倍加速,有的直接翻车。论文标题很直白:Can KV Cache Be Reused Across Model Updates? A DeepSeek-V4-Flash Case Study(arXiv 2611.14051)。

#DeepSeek-V4-Flash#DeepSeek-V4-Flash-Base#DeepSeek-V4.1-Flash#DeepSeek-V4-Flash-1017#GLM-5.3#Qwen3-30B-A3B#Qwen3-30B-A3B-Instruct-2507

Original · 9 min readRead the original
Group chat

No comments yet. Start the conversation.