L3.1 Nemotron Sunfall 71B IQ3_XS

Headlines · 1

1. 5d
   llama.cpp Tensor Splitting Shows Promise Over PCIe 3.0 x1

   A community test running Gemma 4 31B Q5 on two RTX 3060 12GB GPUs reported that switching llama.cpp from layer splitting to tensor splitting raised throughput from 14 to 20 tokens per second, a 42% increase, with the second GPU connected through PCIe 3.0 x1. The author said the exact home configuration was not extensively tested while awaiting a PSU upgrade. A separate test on two RTX 5060 Ti 16GB GPUs with PCIe Gen 3 x8 links recorded 20.7 tokens per second for tensor splitting versus 11.7 for layer splitting during prompt generation, while decoding showed transfer rates of roughly 90–110 MB/s in each direction. The author cautioned that the results do not prove PCIe 3.0 x1 has no performance penalty and that average throughput does not capture latency, synchronization overhead, or short-transfer bursts.

   r/LocalLLaMA · 🔥 0

Experience and discussion from the community

Share my L3.1 Nemotron Sunfall 71B IQ3_XS experienceAsk about L3.1 Nemotron Sunfall 71B IQ3_XS