Janas streams MoE experts from SSD, runs 80B models on a 32GB laptop
Janas is an experimental, GPL-3 open-source inference engine by Maurizio Cammalleri that streams mixture-of-experts weights from an NVMe SSD so models larger than RAM can run on ordinary hardware. Its own janas-bench numbers on a Debian 13 laptop (Intel Core Ultra 9 185H, integrated Arc GPU, 32 GB RAM) put the 48.4 GB Qwen3-Next-80B-A3B at 62 tok/s reading a 1,024-token prompt and 24 tok/s writing, against 16 and 2.8 tok/s for llama.cpp on the same files. Janas reports faster prompt reading and generation than llama.cpp on most of the compared models, with one exception: a 3-bit Qwen3-30B-A3B reads prompts slower than llama.cpp.
According to the project, llama.cpp ran out of memory on the 48.4 GB model once the GPU took part on that laptop, while Janas claims interactive speeds on the same machine.
37 tok/s from Qwen3-Next-80B (a 48 GB model) on a laptop with 32 GB of memory, 24 without drafts, where llama.cpp manages 2.8; reading a prompt, 62 tok/s against 16. Janas-LLM was faster than llama.cpp on every model compared on the same laptop (Debian 13 Linux, Intel Core Ultra 9 185H, its integrated Arc GPU, 32 GB, an NVMe SSD; 9-12 October 2026; the whole machine below):
#Janas#llama.cpp#Qwen3-Next-80B-A3B#Qwen3-Coder-Next#Maurizio Cammalleri#Gemma 4