Cohere adds llama.cpp support for North Micro Vision Instruct vision encoder
Cohere staffer Nick (u/nick_frosst) posted on r/LocalLLaMA that the team recently merged llama.cpp support for the vision encoder of North Micro Vision Instruct, which he said means all of Cohere's latest model architectures are now compatible with llama.cpp; the LLM portion shares the architecture of North Mini Code and Command A+, which were already supported. He also said North Micro Vision is being used as the base for several of Cohere's new releases and invited the community to try it and give feedback. In the same post, Cohere said its Cohere Transcribe speech-to-text model is now available for fine-tuning in Unsloth and Unsloth Studio. The compatibility and capability statements come from Cohere's side, with no independent verification or benchmark data provided.
For the local-inference community, llama.cpp support means Cohere's latest architectures can be loaded on a widely used community runtime rather than only vendor tooling.
| Hey guys, Nick here from Cohere. It's been a minute. We had a great time launching Command A+ and beta-testing North Mini Code with you guys earlier this year, your feedback has really helped make our models and delivery much better. one of the biggest things we heard was that you really wanted llama.cpp support, which makes total sense given the nature of this sub. i didn't want to post in here again until we'd been able to fulfill that promise, and I'm really happy to say that we've made progress. Our team recently merged llama.cpp support for the vision encoder of North Micro Vision Instruct, which means that now all of our latest model architectures are compatible with llama.cpp (the LLM is the same arch as for North Mini Code and Command A+ - already supported thanks to the work of people like you guys). North Micro Vision is being used as the base for several of our new releases, and i'm really psyched to say we've got the support you guys have been wanting. Please feel free to try it out and let us know what you think. It's just one step, but there's only more where that came from. On a slightly similar note, Cohere Transcribe is now available in Unsloth and Unsloth Studio for fine-tuning. it's a crazy good STT model, one of the best out there, so if you're a fan of the guys over at r/unsloth (i know we are), give it a whirl. Thanks again for all your feedback, and let us know what else you want to see from Cohere. submitted by /u/nick_frosst [link] [comments] |
#Cohere#llama.cpp#North Micro Vision Instruct#Cohere Transcribe#Unsloth#Unsloth Studio#North Mini Code#Command A+