Five devices chained to run Qwen3.8 Flash Next at 37–39 tok/s on 60k context
r/LocalLLaMA user StayLameBro says he chained two gaming PCs (an RTX 2060 Super and an RX 9060 XT), an M4 Pro MacBook and two iPhone 17 Pro Maxes over Ethernet and USB-C to serve Qwen3.8 Flash Next, which he describes as a 48-layer mixture-of-experts model with 512 experts per layer. The self-measured decode rates are about 42 tok/s at 16k context, 39–41 tok/s at 32k and 37–39 tok/s at 60k, with prompt reading around 850–900 tok/s on long prompts; he credits pipelined speculation for keeping the five devices from idling, landing about 2.2 tokens per step. The author says the current layout serves 69.6k context, capped by the second PC's 16GB, that a second layout read a single prompt past 164k tokens, and that a full 262k layout is still pending; the code is described as coming to a repo shortly.
The post details a heavily sharded setup where pipelined speculative decoding lets two gaming PCs, a MacBook and two iPhones serve one MoE model that fits on no single machine, with every weight resident in VRAM, pinned RAM or unified memory instead of streaming from disk.