OVHcloud发布Olala mini 7A1B
AI 总结
OVHcloud宣布首个Olala模型家族,并推出Olala mini 7A1B。该模型采用BF16格式,总参数70亿、激活参数10亿,使用结合Differential-TPA与Mamba-3的混合注意力机制,初始上下文长度为32K,并计划以Apache 2.0许可在Hugging Face发布。OVHcloud声称,该模型可运行于8GB显存GPU,在RTX A4500上达到每秒160个词元,在纯CPU平台上达到每秒50个词元。
深度见解
Olala mini 7A1B为低显存环境提供了计划开源的模型选择,其架构、部署规格、训练成本和后续路线图也为开发者提供了可评估的具体信息。
| Olala mini is based on a 7-billion-parameter architecture (7B), with one billion active parameters spread across 256 experts per layer, and hybrid attention combining Differential-TPA and Mamba-3, in BF16 format. Only a fraction of the model is used for each request, making it faster and less memory-hungry. As a direct result, the model runs on a GPU with as little as 8 GB of memory, reaching 160 tokens per second on an RTX A4500, and even without a GPU, on a standard processor, at 50 tokens per second. Its context window currently reaches 32,000 tokens. Olala mini will be available as open source on Hugging Face, under the Apache 2.0 licence. submitted by /u/clem844 [link] [comments] |