Reddit post spotlights Taalas ChatJimmy chip claiming 17,000 tok/s on an 8B Llama
A post in r/LocalLLaMA by user /u/IAM_274 highlights ChatJimmy, a chatbot from startup Taalas that the poster says runs an 8B Llama model at 17,000 tokens per second on a single chip by burning the model into silicon rather than loading it onto the hardware. The poster compares that with an average of about 80 tok/s for the same model on an RTX 5070 — a claimed roughly 213x gap — and states the project is an experimental proof of concept, not an actual product yet. The post also flags a tradeoff: the model cannot be swapped once it is fixed to the chip, so a newer model would mean buying a new chip, and it links a Medium article discussing manufacturing costs and power consumption. All performance figures come from the poster and the cited project material, with no independent measurement supplied.
The post is drawing attention to an approach — printing a model directly into silicon rather than running it as software on general-purpose chips — that its author says is not yet a product, so the claimed speed figures remain unverified.
I found this project that I never saw talked about enough which is absolutley baffling to me. The title is not clickbait, and I can't blame you if you thought it is, because I was in your shoes once.
Meet ChatJimmy. It's a chatbot made by Taalas. And it's not your average chatbot. This is the closest LLMs have come to a physical brain than mere software that runs on your computer.
Though not an increment to intelligence (since it's an 8B model) and that's not the point of the project, this single chip makes the model run faster than if it were run on a multi-GPU datacenter. At a blazing 17k tok/s. For reference, the average speed of running the same model on an RTX 5070 is 80 tok/s. This chip is around 213x faster. If someone told me this was possible a year ago, I'd ask for what they're smoking.
The magic is actually pretty simple behind the scenes. The way Taalas was able to achieve such absurd speeds is by burning the model into silicon. Yep, you heard me right. The LLM is no longer loaded on a chip. The LLM is the chip. All computing power is purely gone straight the model's internal matrix math. The concept on it's own sounds cool as fuck, and seeing it work in practice is even cooler.