Splash Fork Streams Qwen Experts From SSD for Faster Mac Decoding
In a report on r/LocalLLaMA, a user said an approximately 800-line Splash 1.4.0 fork running Qwen3.8-Flash-Next at ISTA GSQ-RCO Q2_0 achieved 48.8 tokens per second on a 24GB M5 MacBook Air by streaming nonresident experts from SSD. The configuration kept 6,800 of 24,576 expert slots in about 8.8GB, reported 256K context support, and delivered about 45 tokens per second in pi agent use. Cache-aware routing reportedly halved SSD reads, while desk-first overlap produced token-for-token identical output in the poster’s 11-case gate test.
The report presents a detailed local-inference optimization that could make large-context Qwen models more practical on memory-constrained Macs, although its performance remains a single user’s unverified report.
Qwen3.8-Flash-Next (ISTA GSQ-RCO Q2_0) + Flash-Next-DFlash2 draft on a MacBook Air M5 (24 GB), on Splash 1.4.0 with a small fork. Experts that don't fit are streamed from the SSD: 6,800 of 24,576 expert slots (~28%) live in ~8.8 GB, the rest is read on demand, 256K context.
Decode
| Setup | code | tool | explain | avg t/s |
|---|---|---|---|---|
| Stock Splash 1.4.0 | 31.6 | 29.5 | 25.4 | 28.8 |
| + cache-aware routing (bias 2, top 4 protected) | 48.6 | 45.7 | 35.1 | 43.1 |
#Qwen3.8-Flash-Next#Splash 1.4.0#ISTA GSQ-RCO Q2_0#Flash-Next-DFlash2#pi#MacBook Air M5