12 Used V100 GPUs Run DeepSeek V4.1 Flash's Full Expert Set
Nick Mermiges expanded his private AI server from eight to 12 used Nvidia V100 GPUs, with four cards on each of three boards. He reports that the system runs all 15,360 DeepSeek V4.1 Flash experts on the cards using 4-bit NVFP4. His measurements show 157–184 tokens per second for a single code stream, a 28.1-second first-token time for a 128K context, and power draw of up to about 2,100W. The reports attribute these figures to the owner and note that answer quality in use was not established.
The reported setup shows a possible way to keep DeepSeek V4.1 Flash's full expert set on local second-hand GPUs while reducing long-context first-token latency. The performance figures are owner-reported, and the reports say they do not establish answer quality in use.
A lawyer has upped the ante by upgrading his private AI server from eight to 12 used Nvidia V100s, which can run every DeepSeek V4.1 Flash expert at 4-bit NVFP4. Performance can reach up to 184 tokens per second for code, and long documents start answering in about half the time as before. He says it can outrun published DGX Spark stacks at decoding a single request, for half the price of three units, based on the $9,450 he paid.
Lawyer with too many GPUs: DeepSeek-V4.1-Flash on 12x V100, all 15,360 experts in NVFP4 on the cards. 157-184 tok/s on one code stream, faster than any published 4x DGX Spark stack and double 3 Sparks, for half the price of 3 Sparks. 128K file read in 28 s. from r/LocalLLM
Some of the best movie franchises come in threes, and Nick Mermiges seems determined to match that, taking his original setup — which he previously upgraded with more memory and other fixes — and expanding it for fuller precision, after a previous upgrade got him most of the way there. The newest upgrade, this time, involves a third board of V100s, for a total of 12 with four per board. There may be a method to his madness.