Create

Sign in to ReadmeX

or

How we handle your data: Privacy Policy

ReadmeX

Your AI reading companion

What would you like to catch up on?

Pick a briefing, or ask your own question.

Your community and people briefings stay personal to you.

Story

Developer reports DeepSeek-V4.1-Flash running on eight V100 GPUs

AI summary

A developer reported running a quantized DeepSeek-V4.1-Flash-1017 build on eight Tesla V100-SXM2 GPUs, with all 248.8 GB of weights held in VRAM and a 524,288-token context limit per request. The developer's measurements showed code generation at 215 tokens per second after a 1K prompt, falling to 160 tokens per second at 256K. Prefill reached about 2,200 tokens per second at 128K and 256K. The server handles one request at a time, and the report publishes launch configurations, A/B test results, raw benchmark data, and tooling.

In-depth insight

The report documents a tuned deployment of a quantized DeepSeek-V4.1-Flash build on eight older V100 GPUs, including workload-specific throughput, long-context tests, and public configurations and benchmark data.

#DeepSeek#DeepSeek-V4.1-Flash-1017#Tesla V100-SXM2#ISTA-DASLab#1Cat-vLLM#Model Runner V2

Original · 4 min readRead the original
Group chat

No comments yet. Start the conversation.