Basalt 0.4.0 adds model routing and Qwen 3.8 27B
The Basalt developer announced that version 0.4.0 adds multi-model and router support, starting with DeepSeek V4.1 Flash. The release also introduces video support, file handling in chats, and various fixes and improvements. Basalt 0.4.0 also features the project's first dense model, Qwen 3.8 27B. The developer reported support for chat, tools, thinking, images, video, and contexts up to 262K. Reported performance included up to 776 tokens per second for structured output, 262 for conversation, 194 for prose, and 12,000 tokens per second for 4K prefill. The developer also reported 92.8% top-1 agreement with the full BF16 model. The announcement said the dense-kernel work was undertaken in preparation for Qwen 4.
| Hi everyone! I wanted to post a quick update on Basalt. Since last I posted, multi model and router support was implemented, starting with DeepSeek V4.1 Flash (IQ1_M 90 tok/s conv, 246 tok/s struct on a 5090 + 5060 Ti). Also new: video support, files in chats, and a bunch of fixes and improvements. Now 0.4.0 is out featuring Basalt's first dense model, Qwen 3.8 27B. It's fast and accurate, top-1 92.8% vs the full BF16 model. This, in preparation for Qwen 4, so we had a headstart on the dense kernels. The numbers: Links: GitHub: https://github.com/jesdga95/basalt Site: https://basalt.jgulfo.com 27B quant: https://huggingface.co/jesdga/Qwen3.8-27B-Basalt Thanks to everyone who has filed issues, sent PRs or posted benchmark rows, special thanks to the sponsors who keep this going, happy to see Basalt gain so much traction in such a very short time! submitted by /u/jesdga95 [link] [comments] |