Qwen3

Headlines · 7

1. 3d
   Waypoint-OPD gates policy-shift distillation with a mid-training teacher

   An arXiv paper introduces Waypoint-OPD, a new take on policy-shift distillation, which reuses the difference between a teacher model's log probabilities before and after reinforcement learning (RL) as supervision on student-generated trajectories. The authors argue that this endpoint shift alone cannot show whether local action preferences evolved consistently during teacher training, since the same endpoint difference can come from early and late shifts that reinforce or partly cancel each other. Waypoint-OPD therefore uses an intermediate teacher checkpoint from the same RL run to compare early and late shifts in relative action preferences at student-visited prefixes, keeping the endpoint shift only when the two point in the same direction and suppressing it when they oppose; this adds just one extra frozen-teacher scoring pass. Using RL-trained Qwen3-4B teachers with Qwen3-8B, Qwen3-14B and Qwen3-30B-A3B students on mathematical and code reasoning, the method posts the highest domain-average score among compared distillation methods in all six domain-student settings, and ablations with the Qwen3-8B student show that removing or shuffling the gates degrades math reasoning.

   Hugging Face · Papers · 🔥 0
2. 3d
   τ-Emotion-Bench exposes LLM agent fragility under emotional users

   A paper accepted to the UserSim Workshop at NeurIPS 2026 introduces τ-Emotion-Bench, a ten-domain benchmark that evaluates tool-calling agents under controlled emotion-conditioned user interactions while preserving the underlying task objective. The authors also present Tracer, a scalable agentic data-generation pipeline that builds policy-valid tasks and collects trajectories, letting dissatisfied users terminate inefficient interactions so trajectories with excessive user dependence are filtered out. Experiments with two user simulators show frontier agents remain unreliable, with substantial gaps between average task success and consistent task completion, while Tracer-generated supervision improves Qwen3 models across scales and transfers to τ-Bench and τ-Trait.

   Hugging Face · Papers · 🔥 0
3. 4d
   CriticOPD adds teacher critiques to on-policy distillation

   A new arXiv paper proposes CriticOPD, a method for improving on-policy distillation (OPD), which trains a student on its own rollouts using token-level teacher supervision. The authors argue that OPD assumes the teacher is "recoverable" — able to reach a correct answer from any prefix the student writes — and show this fails on incorrect rollouts, where the teacher recognizes the error but follows it anyway. CriticOPD has the teacher critique an incorrect rollout so the student can rewrite its solution, with the teacher scoring the output that includes the critique as privileged information while the student trains on the revision without it. Across two Qwen3 student models and six mathematical reasoning benchmarks, CriticOPD improved over OPD, with the largest gains on competition problems; code is linked from the paper page.

   Hugging Face · Papers · 🔥 0
4. 4d
   Latent Compaction compresses KV caches via test-time training

   A new arXiv paper introduces Latent Compaction, a method that learns a compact, document-specific KV cache through test-time training to cut the memory and attention cost of long-context generation. Cache entries are initialized using reconstruction importance, teacher attention and key-space diversity; training first fits attention outputs and log-normalizers to those of the full cache, then jointly optimizes the whole cache via end-to-end distillation from the full-cache model, with model weights frozen and only generic document prompts used, not downstream questions or answers. The authors report gains over budget-matched compression baselines on RULER, LongBench and Phonebook Lookup — for example 96.7% accuracy at 32x compression on RULER versus 40.0% for KVZip+ — and say the method transfers across model scales and families, improving retrieval on Qwen3 and MiniCPM5. Code is released.

   Hugging Face · Papers · 🔥 0
5. 7d
   ByteDance paper finds periodic blind spots in DeepSeek V4 chunked KV-cache compression

   A ByteDance team paper, “Periodic Weak Spots: Phase Sensitivity from Chunked KV-Cache Compression,” says the chunked KV-cache compression used by the DeepSeek V4 series creates “phase sensitivity”: adding only decorative equals signs ahead of a code snippet flipped DeepSeek-V4-Flash-Base's completion between 8 and 32, with an error rate of about 71.3% when the count of equals signs modulo 4 was 0 or 1 and about 91.5% accuracy when it was 2 or 3. In a 128K-token needle-in-a-haystack test with roughly 16,000 key-value pairs, moving a single pair changed accuracy by up to 40.2% for V4-Flash-Base and 34.8% for V4-Pro-Base, with gaps of 19.1% and 14.8% remaining after post-training; halving the compression stride in DeepSeek-V4.1-Flash narrowed the gap to 6.1% but did not remove it. Control models trained from scratch on the Qwen3-0.6B architecture showed the same periodic swings whenever fixed-length chunked compression was used, while a full-attention baseline did not, which the authors present as a structural flaw rather than a tuning issue.

   雷峰网 AI · 🔥 0
6. 7d
   ByteDance Seed finds periodic blind spots in DeepSeek-V4 long-context retrieval

   A ByteDance Seed team with researchers from Princeton, Stanford and UC Berkeley systematically tested DeepSeek-V4 and V4.1 and describes a "phase sensitivity" effect in the paper "Periodic Weak Spots: Phase Sensitivity from Chunked KV-Cache Compression": because CSA compresses the KV cache with overlapping windows of length 8 and stride 4, shifting the same information by a few tokens can sharply change whether it is retrieved. In a 128K needle-in-a-haystack test, the base V4-Flash model scored 69.92% in the best position group versus 29.69% in the worst, a 40.23-point gap; the post-trained V4-Flash-0731 narrowed this to 19.14 points, while V4-Pro showed 34.77 and 14.84 points for base and post-trained versions. The article says DeepSeek had already cut the compression stride from 4 to 2 in V4.1-Flash and added an Engram module, shrinking the gap to 6.1 points without removing the periodicity.

   36氪 人工智能 · 🔥 0
7. Oct 6
   OpenTPU open-sources an AI accelerator built by AI

   FeSens has published OpenTPU, an open-source AI accelerator whose hardware, instruction set, simulator, compiler, and host software are contained in one monorepo. The project reports running models including LFM2.5-230M, Qwen3, Qwen3.5, Gemma 4, SmolLM3, and Phi-4-mini on an FPGA card with bit-for-bit agreement between the simulator and hardware.

   Hacker News · AI（100+ 分） · 🔥 0

Experience and discussion from the community

Share my Qwen3 experienceAsk about Qwen3