NVIDIA GeForce RTX 5060 Ti

Headlines · 1

1. Oct 21
   Developer runs DeepSeek V4.1 Flash IQ1_S on RTX 5090 at 53.6 tok/s

   A developer posting on r/LocalLLaMA reports running DeepSeek V4.1 Flash 1017 at Unsloth's UD-IQ1_S quantization in Basalt, an inference engine they are developing, on an RTX 5090 plus an RTX 5060 Ti 16 GB and 128 GB of DDR5. With all 107 GiB of experts held in RAM and the 5090 used as an expert cache, the setup reached 53.6 tok/s decode and about 610 tok/s prefill, versus roughly 20 tok/s for a llama.cpp PR #30817 baseline on the same hardware and quantization. Letting the 5060 Ti compute some experts raised decode to 63.5 tok/s, but the author reports no perplexity or KL-divergence quality measurements yet, only greedy-match agreement with llama.cpp on test prompts.

   r/LocalLLaMA · 🔥 0

Experience and discussion from the community

Share my NVIDIA GeForce RTX 5060 Ti experienceAsk about NVIDIA GeForce RTX 5060 Ti