Colibri v2.1.0 Brings Tiered MoE Inference to Local Hardware
AI summary
JustVugg released colibri v2.1.0, an open-source C inference engine that places frequently used MoE experts on CUDA or Vulkan GPUs, keeps warm experts in RAM, and streams cold experts from NVMe. The project says the engine can run large open models on existing hardware without requiring a GPU and currently provides 13 engines, including 10 for language models.
In-depth insight
Colibri offers a lightweight, open-source approach to running very large MoE models locally by managing experts across GPU, memory, and storage tiers.
This source is a digest of several news items (“JustVugg/colibri”). Find this story in the original.
#JustVugg#colibri v2.1.0#GLM-5.2#GLM-5.3#DeepSeek V4.1 Flash#Qwen3.6#Qwen-Image-2.1