ByteDance Seed finds DeepSeek-V4 phase sensitivity in long-context retrieval
ByteDance's Seed team reports in a paper that adding a few irrelevant characters ahead of the same prompt makes DeepSeek-V4's answers flip on a 4-token cycle: on a code-completion task, the wrong answer 32 averaged 71.3% probability at some fill lengths while the correct answer 8 rose to 91.5% at others. In a 128K-token needle-in-a-haystack test where the keys, question and total length were held fixed and only the target's position changed, DeepSeek-V4-Flash-Base showed a maximum accuracy gap of 40.2 percentage points and V4-Pro-Base 34.8 points. The team ties the effect to the models' chunked KV cache compression, with the period matching the compression stride, and reproduces it in Qwen3-0.6B-based models trained with several compression schemes, while an uncompressed full-attention baseline did not show the same periodicity. Post-training and newer versions narrow the gap (19.1, 14.8 and 6.1 points for V4-Flash-0731, V4-Pro-0813 and V4.1-Flash-0910 respectively), but it persists.
If it holds, the finding suggests average-score benchmarks can hide position-dependent retrieval weaknesses, so models using chunked KV cache compression should be evaluated phase by phase.
DeepSeek的神鬼二象性被字节Seed团队逮到了。
同一道题,什么都没改,只在前面多塞几个无关紧要的字符,模型突然就不会了???
而且还不是偶尔抽风。
Seed研究人员发现,DeepSeek-V4的表现竟然会随着信息的输入位置,每隔4个Token周期性变化。
啥意思?模型记不记得住一件事,居然还要看这件事在输入里站哪儿??