DeepSeek-V4.1-Flash双DGX Spark实测
PLI Labs 的 Nick Mermiges 使用两台 NVIDIA DGX Spark，对 2.75 bpw 的 DeepSeek-V4.1-Flash-1017 进行了 EXL3 和 vLLM 基准测试。一台配有双 RTX 3090 的 EPYC 主机负责承载 Engram 表并执行推测式起草。作者报告的三次运行中位数显示，256K 提示词预填充速度为 2,618 token/s，四路结构化 JSON 输出为 322 token/s。报告公开了代码、启动脚本、基准工具和原始结果，同时指出散文生成吞吐和解码速度低于部分对比方案。计划中的 1M-token 上下文、llama-benchy、BFCL 以及同场 KLD 对比测试尚未完成，前缀缓存代码也尚未发布。
公开的配置、脚本和原始基准数据，为在紧凑型多 GPU 硬件上运行和评估这一 DeepSeek-V4.1-Flash 部署方案提供了参考。
Update from the lawyer with the V100 server. I went and got two DGX Sparks, because of course I did. The mid-life crisis remains on schedule.
Same deal as before: still a lawyer, still driving the whole thing through Claude Code, and Claude Code also ran the benchmarks, built these tables and helped me write this up. I'm Nick Mermiges, Esq., founder of PLI Labs. The repo below is ours.
The setup: DeepSeek-V4.1-Flash-1017 at 2.75 bpw EXL3, vLLM tensor parallel across two Sparks. The twist is a third box, my EPYC with the 3090s, doing two jobs for the Sparks: - it holds the Engram tables in system RAM and serves them over RDMA; - it runs the DSpark speculative drafter on an NVLinked pair of 3090s.
The memory that frees on the Sparks goes to KV cache: about 1.93M tokens of pool. There's also a prefix cache on the Sparks' NVMe, so a 150K prompt survives a server restart. First token comes back in 2.58 s instead of 58.6 s cold.
Numbers. I re-implemented MiaAI's sparkDash protocol so this would be apples to apples with the most popular 2x Spark build. Sparks at 2300 MHz, 3 reps, medians, today.
#DeepSeek-V4.1-Flash-1015#PLI Labs#NVIDIA DGX Spark#NVIDIA RTX 3090#vLLM#EXL3#Claude Code