Gemma 4 31B Q5
AI overview
Sign in and the AI will write an overview from our coverage.
Headlines · 1
- llama.cpp Tensor Splitting Shows Promise Over PCIe 3.0 x1
A community test running Gemma 4 31B Q5 on two RTX 3060 12GB GPUs reported that switching llama.cpp from layer splitting to tensor splitting raised throughput from 14 to 20 tokens per second, a 42% increase, with the second GPU connected through PCIe 3.0 x1. The author said the exact home configuration was not extensively tested while awaiting a PSU upgrade. A separate test on two RTX 5060 Ti 16GB GPUs with PCIe Gen 3 x8 links recorded 20.7 tokens per second for tensor splitting versus 11.7 for layer splitting during prompt generation, while decoding showed transfer rates of roughly 90–110 MB/s in each direction. The author cautioned that the results do not prove PCIe 3.0 x1 has no performance penalty and that average throughput does not capture latency, synchronization overhead, or short-transfer bursts.
r/LocalLLaMA · 🔥 0
Experience and discussion from the community
Share my Gemma 4 31B Q5 experienceAsk about Gemma 4 31B Q5
Nobody has shared their experience with Gemma 4 31B Q5 yet.