律师用八块二手V100搭建本地AI服务器
据报道,一名律师使用八块二手 Nvidia Tesla V100-SXM2 32GB 显卡搭建办公室内部AI服务器,以避免案件资料离开办公场所。Tom’s Hardware 转述称,该系统由两组四卡 NVLink 主板组成,可将 DeepSeek V4.1 Flash 的专家模块全部载入显存;构建者实测代码生成最高达每秒215个词元,法律写作最高达每秒118个词元。目前服务器用于案件文件的事实提取、摘要和初步起草,但所有输出都需人工复核后才能外发。
这一案例展示了将二手数据中心硬件与本地模型推理用于隐私敏感型法律工作的可行路径,但性能与成本节省均来自个人实测,尚未得到独立验证。
A lawyer posted on Reddit about their private AI server built from eight used 2018-era Nvidia Tesla V100 data center GPUs so case files never have to leave the office. With two 4-card SXM2 NVLink boards, four PEX switches, and SlimSAS cables everywhere — plus one burnt power connector — the builder now runs the open DeepSeek V4.1 Flash model with every expert in GPU memory, at up to 215 tokens per second. The four-GPU boards cost about $2,400 apiece when the builder bought them.
Lawyer's V100 homelab, update: two 4-card SXM2 NVLink boards, four PEX switches, SlimSAS everywhere, one burnt power connector, and a 250 GB model running entirely in VRAM at 200+ tok/s from r/homelab
The eight Tesla V100-SXM2-32GB cards, each with 32GB of HBM2 at 900 GB/s, plug into SXM2, Nvidia’s mezzanine socket for data center GPUs, rather than PCIe slots, and are split across two baseboards. Inside each board, all four cards are NVLinked — NVLink is Nvidia’s direct GPU-to-GPU link — for 128GB per board, but the two boards only communicate over PCIe. Each board connects to the host through two PEX (PLX) switch cards over SlimSAS cables, four switches in all, with each switch handling two GPUs at PCIe 3.0 x8 speeds over its own x16 uplink.
#Nvidia#DeepSeek#DeepSeek V4.1 Flash#Tesla V100-SXM2 32GB#Gigabyte Threadripper Pro 3945WX#DGX Spark