llama.cpp Tensor Splitting Shows Promise Over PCIe 3.0 x1
A community test running Gemma 4 31B Q5 on two RTX 3060 12GB GPUs reported that switching llama.cpp from layer splitting to tensor splitting raised throughput from 14 to 20 tokens per second, a 42% increase, with the second GPU connected through PCIe 3.0 x1. The author said the exact home configuration was not extensively tested while awaiting a PSU upgrade. A separate test on two RTX 5060 Ti 16GB GPUs with PCIe Gen 3 x8 links recorded 20.7 tokens per second for tensor splitting versus 11.7 for layer splitting during prompt generation, while decoding showed transfer rates of roughly 90–110 MB/s in each direction. The author cautioned that the results do not prove PCIe 3.0 x1 has no performance penalty and that average throughput does not capture latency, synchronization overhead, or short-transfer bursts.
The preliminary community measurements offer practical guidance for multi-GPU llama.cpp setups, while the untested home configuration, use of PCIe x8 in the separate experiment, and unresolved latency and synchronization effects limit the conclusion.
#llama.cpp#Gemma 4 31B Q5#L3.1 Nemotron Sunfall 71B IQ3_XS#NVIDIA RTX 3060#NVIDIA RTX 5060 Ti#Intel Core i5-14400F