NVIDIA RTX 5070

Headlines · 1

1. Oct 21
   Reddit post spotlights Taalas ChatJimmy chip claiming 17,000 tok/s on an 8B Llama

   A post in r/LocalLLaMA by user /u/IAM_274 highlights ChatJimmy, a chatbot from startup Taalas that the poster says runs an 8B Llama model at 17,000 tokens per second on a single chip by burning the model into silicon rather than loading it onto the hardware. The poster compares that with an average of about 80 tok/s for the same model on an RTX 5070 — a claimed roughly 213x gap — and states the project is an experimental proof of concept, not an actual product yet. The post also flags a tradeoff: the model cannot be swapped once it is fixed to the chip, so a newer model would mean buying a new chip, and it links a Medium article discussing manufacturing costs and power consumption. All performance figures come from the poster and the cited project material, with no independent measurement supplied.

   r/LocalLLaMA · 🔥 0

Experience and discussion from the community

Share my NVIDIA RTX 5070 experienceAsk about NVIDIA RTX 5070