Mica

Headlines · 1

1. Oct 21
   Mica v0.5: a 4B decision model that runs on one consumer GPU

   A developer on r/LocalLLaMA released Mica v0.5, a 4B decision model that runs on a single consumer GPU and follows the earlier Mica v0.1. In self-reported measurements on one RTX 6000 Ada, the developer says it takes 19.8 ms per decision at 256 tokens and about 1.2 s at 32k tokens, claiming it is up to 3.8x faster than the other 4B judges measured, 12 to 19x faster than Perplexity's 27B decider and 25 to 340x faster than API decision models. The developer reports 84/111 on JevBench Hard, described as the best of any 4B model found with only Microsoft's 9B Decision-1 higher, and a JDI score of 58.8, while acknowledging that top 4B models on JDI are benchmark-focused and that training used data from the areas the benchmarks cover. Mica v0.5 handles yes/no, multiple choice and scores from 2 to 255 levels, accepts inputs up to 32k tokens, and is released under Apache-2.0 with INT8 and GGUF files, running on 8 GB cards and up.

   r/LocalLLaMA · 🔥 0

Experience and discussion from the community

Share my Mica experienceAsk about Mica