Reddit 用户分享双 M5U 集群的 GLM 与 Qwen 数据
一名 Reddit 用户称，在两台各配备 256GB 内存、80 核的 M5U 上运行了 GLM 5.3 q6 和 Qwen Flash Next q6。帖子称 GLM 的预填充速度为 4.9k、解码速度为每秒 50 个 token，而通过 Tensorfold 运行的 Qwen 达到 8k 预填充和每秒 200 个 token；由于预填充数据来自大模型的解释，作者明确表示无法确认其准确性。作者称 DeepSeek 根据教程配置了 oMLX，并向社区征求优化建议。
尽管这些数据尚未验证，但它为潜在 M5U 买家提供了按模型划分的本地推理实测参考，同时也清楚交代了配置方法与测量局限。
Caveats: IDK how to set anything up and I asked Deepseek to handle it - not enough training data so without specific guides the set up doesn't work well. I'm not a programmer. My previous model was Qwen 3.8 27B q4 using ninfer. Was getting around 1.4k prefill and 100-120 tokens decode.
Just want to provide data for anyone interested in purchasing M5U 256gb (80 cores) x 2. And if you have suggestions on how to optimize my set up I'd really appreciate it.
- GLM 5.3 q6 cluster: prefill 4.9k (idk if this is right but this is what the LLM says, two thunderbolt 5 cables), decode: 50 tok/s. oMLX set up with Deepseek API.
- Qwen Flash Next q6 cluster: prefill 8k and decode 200 using Tensorfold. I copied their guide, sent it to Deepseek, and asked it to set it up.
GLM was too slow for me so I am using Qwen Flash Next.
I am using Hermes as the harness (hermes has deepseek harness as research agent and Pi harness for coding agent). I am wondering whether using a different harness with less context overhead would make things faster or if since it is mostly context it doesn't really make a difference.
submitted by /u/Viktri1 [link] [comments]
#M5U#GLM 5.3 q6#Qwen Flash Next q6#Qwen 3.8 27B q4#oMLX#Tensorfold#Hermes#DeepSeek API#n Infer