创建

登录 ReadmeX

或

了解我们如何处理你的信息: 隐私政策

ReadmeX

陪你读懂 AI 动态

今天,想先了解什么?

选一份动态总结,或直接聊聊你的问题。

社区和关注动态总结只属于你自己。

资讯

12张二手V100运行DeepSeek V4.1 Flash完整专家集

AI 总结

Nick Mermiges将私人AI服务器从8张扩展至12张二手Nvidia V100 GPU,每块主板配备4张显卡。他称,系统使用4-bit NVFP4将DeepSeek V4.1 Flash的全部15,360个专家驻留在显卡上。其自测显示,单代码流速度为157–184词元每秒,128K上下文首词元耗时28.1秒,功耗最高约2,100W。报道将这些数据归于机主,并指出这些数字未能证明实际使用中的答案质量。

深度见解

据报道,该配置提供了一种将DeepSeek V4.1 Flash完整专家集保留在本地二手GPU上、同时缩短长上下文首词元等待时间的可能。性能数据来自机主自测,报道也指出这些数据不能证明实际使用中的答案质量。

A lawyer has upped the ante by upgrading his private AI server from eight to 12 used Nvidia V100s, which can run every DeepSeek V4.1 Flash expert at 4-bit NVFP4. Performance can reach up to 184 tokens per second for code, and long documents start answering in about half the time as before. He says it can outrun published DGX Spark stacks at decoding a single request, for half the price of three units, based on the $9,450 he paid.

Lawyer with too many GPUs: DeepSeek-V4.1-Flash on 12x V100, all 15,360 experts in NVFP4 on the cards. 157-184 tok/s on one code stream, faster than any published 4x DGX Spark stack and double 3 Sparks, for half the price of 3 Sparks. 128K file read in 28 s. from r/LocalLLM

Some of the best movie franchises come in threes, and Nick Mermiges seems determined to match that, taking his original setup — which he previously upgraded with more memory and other fixes — and expanding it for fuller precision, after a previous upgrade got him most of the way there. The newest upgrade, this time, involves a third board of V100s, for a total of 12 with four per board. There may be a method to his madness.

#Nvidia V100#DeepSeek V4.1 Flash#Nvidia DGX Spark#NVFP4

原文约 4 分钟读完阅读原文
群聊

还没有评论,来说说你的看法。