OVHcloud unveils Olala mini 7A1B AI model
AI summary
OVHcloud announced its first Olala model family and introduced Olala mini 7A1B, a BF16 model with 7 billion total parameters and 1 billion active parameters. It uses hybrid attention combining Differential-TPA and Mamba-3, supports a 32K-token context window, and is planned for publication on Hugging Face under the Apache 2.0 license. OVHcloud claims it can run on an 8GB GPU, reach 160 tokens per second on an RTX A4500, and achieve 50 tokens per second on a CPU.
In-depth insight
Olala mini 7A1B adds an open-source AI model planned for lower-memory hardware, while its architecture, deployment targets, training cost and roadmap provide a concrete release for practitioners to evaluate.
| Olala mini is based on a 7-billion-parameter architecture (7B), with one billion active parameters spread across 256 experts per layer, and hybrid attention combining Differential-TPA and Mamba-3, in BF16 format. Only a fraction of the model is used for each request, making it faster and less memory-hungry. As a direct result, the model runs on a GPU with as little as 8 GB of memory, reaching 160 tokens per second on an RTX A4500, and even without a GPU, on a standard processor, at 50 tokens per second. Its context window currently reaches 32,000 tokens. Olala mini will be available as open source on Hugging Face, under the Apache 2.0 licence. submitted by /u/clem844 [link] [comments] |