A dual-RTX 5060 Ti local LLM inference setup
A Reddit user reports running Qwen3.8-27B NVFP4 across two RTX 5060 Ti 16GB GPUs with capy-flash, OpenCode, tensor parallelism, MTP speculative decoding, and an int8 KV cache. The author says connecting both GPUs directly to CPU lanes reduced time to first token by about 2.5 times and doubled batching throughput compared with a chipset-connected card. Their reported configuration reaches 70–95 tokens per second for decoding, while Gen4 x8/x8 performance figures remain estimates.
The report offers concrete hardware and software guidance for local LLM inference, including measured comparisons and context-dependent throughput data, but it comes from a single user without independent replication.
Over the last few months I've been switching harnesses and engines until I finally landed on something good. I started with LM Studio + the Copilot extension and ended up here, so here are some tips for anyone running two 5060 Tis. Prefill is painful on these cards, so for hardware, make sure you have:
A motherboard with both GPUs on CPU lanes, either two slots wired to the CPU or one x16 slot with PCIe bifurcation (x8/x8). My second card used to hang off the chipset; moving it to CPU lanes cut time-to-first-token by about 2.5x and doubled my batching throughput.
A separate low-profile GPU just for system graphics. If one of your 5060 Tis drives the display, it burns 2-3 GB of VRAM, and since the layer split is even, the other card loses the same amount too. That's context you don't get.
Model: these are Blackwell cards, so an NVFP4 quant gives you better prefill and decode.
What I run: capy-flash (formerly ninfer-tp2) + OpenCode, Qwen3.8-27B NVFP4 with tensor parallel across both cards, MTP speculative decoding and int8 KV cache. I get 70-95 t/s decode and ~1,500 t/s prefill at 176K context (~3,500-4,000 t/s on shorter prompts). You can go to full context, I just don't have a third card.
#NVIDIA GeForce RTX 5060 Ti#capy-flash#OpenCode#Qwen3.8-27B NVFP4#AMD Ryzen 7 5800X