Modded 4× Tesla V100 rig beats two DGX Sparks and M5 Ultra in local LLM inference
YouTuber Jun Yamog fitted four 2017-era NVIDIA Tesla V100 cards (32GB HBM2, 900GB/s bandwidth) onto an NVLink carrier board and used milled-down A100 heatsinks to cool them, then benchmarked local inference with a 4-bit quantized Qwen3.8-Flash-Next. The quad-V100 setup processed long prompts roughly 40% faster than two DGX Sparks and about 230% faster than a single M5 Ultra, and generated 100.6 tok/s on short prompts versus 44.5 tok/s for the M5 Ultra, about 121% faster than the DGX Spark pair. A separate eight-GPU RTX PRO 6000 configuration still led most of the runs, and Yamog noted Apple Silicon's gap may largely reflect a lack of software optimization.
It shows that older data-center GPUs can still handle local LLM inference cheaply on the secondhand market, while underscoring how much these comparisons depend on configuration and software tuning.