Betelgeuse Runs Qwen 3.8 Flash Next on Budget Hardware
AI summary
Developer /u/zyxciss reported on r/LocalLLaMA that Betelgeuse, a llama.cpp fork, ran the 68GB Qwen 3.8 Flash Next MoE model on an RTX 3060 12GB with 16GB DDR4 RAM. The developer recorded 16.80 tokens per second cold, 22.03 warm, and bursts up to 48 tokens per second, while claiming byte-exact output without pruning any routed experts. Betelgeuse uses a VRAM expert-slot pool, a RAM victim cache, and io_uring paging to reduce storage reads.
In-depth insight
If independently reproduced, Betelgeuse could make a large MoE model usable on consumer hardware without sacrificing output parity, offering a meaningful alternative to stock llama.cpp and Strata.