DeepSeek-V4

Headlines · 1

1. 14h
   ByteDance Seed finds DeepSeek-V4 phase sensitivity in long-context retrieval

   ByteDance's Seed team reports in a paper that adding a few irrelevant characters ahead of the same prompt makes DeepSeek-V4's answers flip on a 4-token cycle: on a code-completion task, the wrong answer 32 averaged 71.3% probability at some fill lengths while the correct answer 8 rose to 91.5% at others. In a 128K-token needle-in-a-haystack test where the keys, question and total length were held fixed and only the target's position changed, DeepSeek-V4-Flash-Base showed a maximum accuracy gap of 40.2 percentage points and V4-Pro-Base 34.8 points. The team ties the effect to the models' chunked KV cache compression, with the period matching the compression stride, and reproduces it in Qwen3-0.6B-based models trained with several compression schemes, while an uncompressed full-attention baseline did not show the same periodicity. Post-training and newer versions narrow the gap (19.1, 14.8 and 6.1 points for V4-Flash-0731, V4-Pro-0813 and V4.1-Flash-0910 respectively), but it persists.

   36氪 人工智能 · 🔥 14

Experience and discussion from the community

Share my DeepSeek-V4 experienceAsk about DeepSeek-V4