Janas

Headlines · 2

1. Oct 22
   Developer claims unreleased ToRc engine runs ~100GB Qwen model on two £300 ARM mini PCs

   A developer posting on r/LocalLLaMA says an inference engine he is writing, codenamed ToRc and not yet released, reached 315 tok/s prompt read and 43 tok/s generation with Qwen3.6 35B A3B across two Snapdragon X mini PCs joined by a single USB4 cable, hardware he says cost about £600 in total. The same post reports 46 tok/s read and 20 tok/s generation for a roughly 95GB Qwen3.8 Flash Next IQ4_NL quant on the two boxes, versus 10.9/5.8 tok/s for llama.cpp over RPC, and 34.3/18.3 tok/s for a larger UD-Q4_K_XL quant. The author, who previously contributed ARM porting work to the Janas engine, says ToRc uses CPU cores plus the Hexagon NPU with no GPU involved; all figures are his own measurements and have not been independently verified.

   r/LocalLLaMA · 🔥 0
2. Oct 12
   Janas streams MoE experts from SSD, runs 80B models on a 32GB laptop

   Janas is an experimental, GPL-3 open-source inference engine by Maurizio Cammalleri that streams mixture-of-experts weights from an NVMe SSD so models larger than RAM can run on ordinary hardware. Its own janas-bench numbers on a Debian 13 laptop (Intel Core Ultra 9 185H, integrated Arc GPU, 32 GB RAM) put the 48.4 GB Qwen3-Next-80B-A3B at 62 tok/s reading a 1,024-token prompt and 24 tok/s writing, against 16 and 2.8 tok/s for llama.cpp on the same files. Janas reports faster prompt reading and generation than llama.cpp on most of the compared models, with one exception: a 3-bit Qwen3-30B-A3B reads prompts slower than llama.cpp.

   Hacker News · AI（100+ 分） · 🔥 0

Experience and discussion from the community

Share my Janas experienceAsk about Janas