开发者发布Qwen3-0.6B MoE转换版
开发者bayliner1980发布Qwen3-0.6B-MoE-A0.5B，即Qwen3-0.6B的混合专家模型转换版，并同步提供GGUF版本。开发者将Qwen 2.5的qwen2moe方法适配至Qwen3的qwen3moe，还以伪共享专家弥补因Qwen3缺少共享专家而损失的部分质量。其自测显示，该版本在所有质量指标上均逊于稠密模型，在这一规模下GPU速度没有提升；转换后模型还存在代码能力损坏、思考模式未训练等问题。
该项目提供了可下载的GGUF构建和开发者自测数据，但质量退化且缺乏独立验证，实用价值受到限制。
This is another update about my MoE-fication project, the first post can be found here.
Foreword
Since my last model conversion, I’ve been experimenting a lot with new methods, though sadly, most attempts haven't helped much. Compatibility has been the main limiting factor. I’ve made GGUF compatibility a priority to ensure as many people as possible can use my models, but this means sacrificing some potential improvements.
Still, my perfectionism has finally let me publish another model: Qwen3-0.6B, my first chat-style conversion. This brought two new challenges: preserving its chat capabilities during conversion and adapting my methods from Qwen 2.5’s qwen2moe to Qwen3’s qwen3moe.
The latter proved particularly difficult. Unlike the previous architecture, Qwen3 has no shared expert, which helped preserve quality in my earlier conversions by keeping important neurons always active. Without it, quality dropped considerably.
To mitigate this, I experimented with a pseudo-shared expert, basically an expert that the router always picks. It doesn’t work quite as well as a true shared expert due to differences in weighting, but it does recover some of the lost quality.
#Qwen3-0.6B-MoE-A0.5B#Qwen3-0.6B#Qwen#bayliner1980#GGUF#qwen3moe