Basalt 0.4.0 加入模型路由与 Qwen 3.8 27B
AI 总结
Basalt 开发者宣布,0.4.0 版本新增多模型与路由支持,首个支持的模型是 DeepSeek V4.1 Flash。此次发布还加入视频支持、聊天文件处理以及多项修复和改进。 Basalt 0.4.0 还推出了该项目的首个稠密模型 Qwen 3.8 27B。开发者称,该模型支持聊天、工具调用、思考、图像、视频以及最高 262K 的上下文。公布的性能数据包括结构化输出最高 776 tokens/s、对话 262 tokens/s、散文 194 tokens/s,以及 4K 预填充 12,000 tokens/s。开发者还称,其与完整 BF16 模型的 top-1 一致率为 92.8%。公告表示,相关稠密内核工作是为 Qwen 4 做准备。
| Hi everyone! I wanted to post a quick update on Basalt. Since last I posted, multi model and router support was implemented, starting with DeepSeek V4.1 Flash (IQ1_M 90 tok/s conv, 246 tok/s struct on a 5090 + 5060 Ti). Also new: video support, files in chats, and a bunch of fixes and improvements. Now 0.4.0 is out featuring Basalt's first dense model, Qwen 3.8 27B. It's fast and accurate, top-1 92.8% vs the full BF16 model. This, in preparation for Qwen 4, so we had a headstart on the dense kernels. The numbers: Links: GitHub: https://github.com/jesdga95/basalt Site: https://basalt.jgulfo.com 27B quant: https://huggingface.co/jesdga/Qwen3.8-27B-Basalt Thanks to everyone who has filed issues, sent PRs or posted benchmark rows, special thanks to the sponsors who keep this going, happy to see Basalt gain so much traction in such a very short time! submitted by /u/jesdga95 [link] [comments] |