Qwen3.8-Flash-Next 125B benchmarked on 24GB Apple Silicon Macs
A Reddit contributor benchmarked the 76GB-on-disk Qwen3.8-Flash-Next 125B GSQ-RCO IQ3_XXS quantization with the MoEShift backend on 24GB M4 Pro and M5 Pro Macs. Median results across three runs with prompt caching disabled included 374.15 tok/s prefill and 14.69 tok/s short-prompt decode on the M5 Pro, with peak physical process memory of 8.5GB on both systems.
The community measurements show that a 125B mixture-of-experts model can run with useful throughput and an 8.5GB process footprint on 24GB Apple Silicon Macs by offloading weights to SSD.
I benchmarked Qwen3.8-Flash-Next 125B ISTA-DASLab's GSQ-RCO IQ3_XXS quant (76GB on disk), on two Macs with 24GB unified memory using MoEShift backend:
| Metric | M4 Pro (medians) | M5 Pro (medians) |
|---|---|---|
| Short prompt decode (72 input tokens) | 9.84 tok/s | 14.69 tok/s |
| Long prompt prefill (6,060 input tokens) | 199.09 tok/s | 374.15 tok/s |
| Long prompt decode (512 output tokens) | 9.57 tok/s | 13.14 tok/s |
| Long prompt inference time | 83.9s | 55.1s |
| Peak physical process memory | 8.5GB | 8.5GB |
These are medians of 3 runs using the MoEShift community benchmark runner (prompt caching disabled). The results are particularly interesting for prefill making this model perfectly usable in a chat or in an agent, while both machines stayed within 8.5GB peak process footprint. This is a very promising benchmark even for the upcoming Qwen4 MoE.
#Qwen3.8-Flash-Next 125B#MoEShift#ISTA-DASLab#GSQ-RCO IQ3_XXS#Apple M4 Pro#Apple M5 Pro