Create

Sign in to ReadmeX

or

How we handle your data: Privacy Policy

ReadmeX

Your AI reading companion

What would you like to catch up on?

Pick a briefing, or ask your own question.

Your community and people briefings stay personal to you.

Story

Lawyer Runs DeepSeek V4.1 Flash on Eight Used Tesla V100 GPUs

AI summary

A lawyer reportedly built an in-office AI server from eight used 2018-era Nvidia Tesla V100-SXM2 32GB GPUs to keep case files from leaving the office. According to Tom’s Hardware, the two four-GPU NVLink boards run DeepSeek V4.1 Flash with its experts in GPU memory, reaching up to 215 tokens per second for code and 118 tokens per second for legal writing in the builder’s measurements. The system is used for first-pass document analysis and routine drafting, with human review required before any output leaves the office.

In-depth insight

This provides a practical, source-reported example of combining used data-center hardware with local model inference for privacy-sensitive legal work, although its performance and cost savings have not been independently verified.

A lawyer posted on Reddit about their private AI server built from eight used 2018-era Nvidia Tesla V100 data center GPUs so case files never have to leave the office. With two 4-card SXM2 NVLink boards, four PEX switches, and SlimSAS cables everywhere — plus one burnt power connector — the builder now runs the open DeepSeek V4.1 Flash model with every expert in GPU memory, at up to 215 tokens per second. The four-GPU boards cost about $2,400 apiece when the builder bought them.

Lawyer's V100 homelab, update: two 4-card SXM2 NVLink boards, four PEX switches, SlimSAS everywhere, one burnt power connector, and a 250 GB model running entirely in VRAM at 200+ tok/s from r/homelab

The eight Tesla V100-SXM2-32GB cards, each with 32GB of HBM2 at 900 GB/s, plug into SXM2, Nvidia’s mezzanine socket for data center GPUs, rather than PCIe slots, and are split across two baseboards. Inside each board, all four cards are NVLinked — NVLink is Nvidia’s direct GPU-to-GPU link — for 128GB per board, but the two boards only communicate over PCIe. Each board connects to the host through two PEX (PLX) switch cards over SlimSAS cables, four switches in all, with each switch handling two GPUs at PCIe 3.0 x8 speeds over its own x16 uplink.

#Nvidia#DeepSeek#DeepSeek V4.1 Flash#Tesla V100-SXM2 32GB#Gigabyte Threadripper Pro 3945WX#DGX Spark

Original · 5 min readRead the original
Group chat

No comments yet. Start the conversation.