Qwen3-30B-A3B

Headlines · 4

1. 3d
   llama.cpp PR fuses MoE matrix multiplications and GLU in MMQ

   A llama.cpp pull request proposes fusing MoE gate/up matrix multiplications and GLU in the CUDA MMQ path. The PR reports prompt-processing gains of 5.7%–17.2% across DGX Spark and RTX 5090 systems using several Qwen MoE model and quantization configurations. The supplied report does not establish that the change has been merged or released, and the benchmark figures have not been independently verified.

   r/LocalLLaMA · 🔥 0
2. Oct 23
   Prime Intellect extends Mega MoE for distributed training

   Prime Intellect says it extended DeepSeek's forward-only Mega MoE kernel with an overlapping backward pass, making the fused MoE implementation usable for distributed training. The vendor reports that its Prime Mega MoE kernels run up to 3.11 times faster than the default PyTorch implementation, while end-to-end training reaches 1.34 times the Prime-RL baseline throughput. Benchmarks used H200 GPUs with Qwen3-30B-A3B and Qwen3-235B-A22B in FP8 configurations.

   Prime Intellect Blog · 🔥 0
3. Oct 14
   Waypoint-OPD gates RL policy-shift distillation with an intermediate teacher checkpoint

   The arXiv paper Waypoint-OPD addresses policy-shift distillation, which reuses the difference between a teacher model's log probabilities before and after reinforcement learning (RL) as supervision on student-generated trajectories. The authors argue that the endpoint shift alone cannot show whether local action preferences evolved consistently during teacher training, since the same endpoint shift can come from early and late shifts that reinforce or partially cancel each other. Their method uses an intermediate teacher checkpoint from the same RL run to compare early and late shifts in relative action preferences at student-visited prefixes, gating the endpoint shift according to directional agreement and suppressing it when the two point in opposing directions. Using RL-trained Qwen3-4B teachers with Qwen3-8B, Qwen3-14B and Qwen3-30B-A3B students on mathematical and code reasoning, the authors report the highest domain-average score among compared distillation methods in all six domain–student settings, and ablations with the Qwen3-8B student show that removing or shuffling gates reduces mathematical reasoning performance.

   Hugging Face · Papers · 🔥 0
4. Oct 9
   Blog SGLang and Miles on NVIDIA Vera Rubin We had early access to two NVIDIA Vera Rubin nodes (8 GPUs) and used them to bring SGLang and Miles up on the platform: SGLang for inference and Miles for RL training. SGLang & Miles Team October 9, 2026

   On this page Summary Inference: Serving Kimi K3 NVFP4 Attention MoE and communication kernels KDA RL training: Miles on NVIDIA Vera Rubin Agentic RL on the Vera CPU What's Next Acknowledgements Summary We had early access to two NVIDIA Vera Rubin nodes (8 GPUs) and used them to bring SGLang and Miles up on the platform: SGLang for inference and Miles for RL training. On the inference side, we tuned SGLang for Kimi K3 NVFP4: this post covers what we changed in the attention, MoE/communication and KDA kernels, and how much each change bought. On the training side, Miles, which uses SG

   LMSYS Blog · 🔥 0

Experience and discussion from the community

Share my Qwen3-30B-A3B experienceAsk about Qwen3-30B-A3B