Maurizio Cammalleri

Headlines · 1

1. Oct 12
   Janas streams MoE experts from SSD, runs 80B models on a 32GB laptop

   Janas is an experimental, GPL-3 open-source inference engine by Maurizio Cammalleri that streams mixture-of-experts weights from an NVMe SSD so models larger than RAM can run on ordinary hardware. Its own janas-bench numbers on a Debian 13 laptop (Intel Core Ultra 9 185H, integrated Arc GPU, 32 GB RAM) put the 48.4 GB Qwen3-Next-80B-A3B at 62 tok/s reading a 1,024-token prompt and 24 tok/s writing, against 16 and 2.8 tok/s for llama.cpp on the same files. Janas reports faster prompt reading and generation than llama.cpp on most of the compared models, with one exception: a 3-bit Qwen3-30B-A3B reads prompts slower than llama.cpp.

   Hacker News · AI（100+ 分） · 🔥 0

Experience and discussion from the community

Share my Maurizio Cammalleri experienceAsk about Maurizio Cammalleri