Volundr 32B

Headlines · 1

1. Oct 19
   Blockway's Agens Engine keeps a 32B model's 262K context within a 16GB GPU budget

   Blockway, a small team from Hong Kong, says 54 of the 72 layers in its 32B Volundr model keep a small fixed-size state instead of a KV cache that grows with every token, and that it built its own MIT-licensed Agens Engine so long context can spill to system RAM within a chosen GPU memory budget. In an uncut demo, a roughly 110K-token novel reads cold in about 69 seconds and a follow-up is answered in 0.5 seconds, with GPU memory peaking around 15.6 GiB; the team discloses that the run was recorded on a 48GB card with the engine held to a 16 GiB plan because it does not yet own a real 16GB card. Blockway also cites Hugging Face user mmssix, who measured 86.4–106.7 tok/s on a 24GB card, and lists an MoE variant, same-GPU image generation, Windows and macOS support, and a compact desktop as future work.

   r/LocalLLaMA · 🔥 0

Experience and discussion from the community

Share my Volundr 32B experienceAsk about Volundr 32B