Qwen3-14B

Headlines · 3

1. 5d
   Waypoint-OPD gates RL policy-shift distillation with an intermediate teacher checkpoint

   The arXiv paper Waypoint-OPD addresses policy-shift distillation, which reuses the difference between a teacher model's log probabilities before and after reinforcement learning (RL) as supervision on student-generated trajectories. The authors argue that the endpoint shift alone cannot show whether local action preferences evolved consistently during teacher training, since the same endpoint shift can come from early and late shifts that reinforce or partially cancel each other. Their method uses an intermediate teacher checkpoint from the same RL run to compare early and late shifts in relative action preferences at student-visited prefixes, gating the endpoint shift according to directional agreement and suppressing it when the two point in opposing directions. Using RL-trained Qwen3-4B teachers with Qwen3-8B, Qwen3-14B and Qwen3-30B-A3B students on mathematical and code reasoning, the authors report the highest domain-average score among compared distillation methods in all six domain–student settings, and ablations with the Qwen3-8B student show that removing or shuffling gates reduces mathematical reasoning performance.

   Hugging Face · Papers · 🔥 0
2. Oct 10
   Two fixed opening tokens lift base model math accuracy to RL level, MIT study finds

   A paper by researchers at MIT, UC Berkeley, the University of Washington and the Allen Institute for AI (Ai2) reports that pre-fixing just the first two tokens of a base model's answer can sharply raise math reasoning accuracy. On MATH-500, Olmo-3-7B rose from roughly 42% to about 78% when forced to start with “.\n\nOkay”, beating its RL-trained counterpart at about 75%, while Qwen3-14B improved from 72% to 87% with “ Alright,” — all without changing any model parameters. The authors also found the largest pre/post-RL prediction shift sits in the first two tokens, and that retraining a model with “Chicken” swapped in for “Okay” in reasoning-adjacent data made “Chicken” work as a reasoning cue too.

   智东西 · 🔥 0
3. Oct 7
   SparseDecoding: decoding-aware pruning for faster LLM inference

   The paper introduces SparseDecoding, a training-free network pruning framework aimed at the decoding stage of LLM inference. It builds calibration matrices from layer-wise activations collected during dense-model autoregressive generation (excluding prefill) so the pruning objective matches decoding activations, and pairs this with an N:M sparse matrix-vector kernel using bitmask indexing and fixed-step traversal. The authors report that on Llama-3.1-8B, Llama-3.3-70B and Qwen3-14B/32B it outperforms fixed-text calibration on long-form generation benchmarks, with up to 1.48x end-to-end wall-clock decoding speedup on A100 GPUs.

   Hugging Face · Papers · 🔥 0

Experience and discussion from the community

Share my Qwen3-14B experienceAsk about Qwen3-14B