DeepSeek Open-Sources the DeepInfer Production Inference Framework
DeepSeek has open-sourced DeepInfer, the inference framework used for high-throughput, low-latency production serving of DeepSeek-V4.1-Flash. The release includes distributed inference, speculative decoding, SWA Bounded Replay, structured output and the deepseek-recipe component for converting between API protocols and model tokens. DeepSeek said the system supports NVIDIA SM100 Blackwell GPUs and Huawei Ascend 950 NPUs; older NVIDIA architectures were not listed. According to DeepSeek, one cluster processes 155 trillion tokens per day and generates $38 in revenue per GPU-hour at off-peak pricing. The release does not include all of DeepSeek’s online infrastructure, as KVCache management and cross-replica load-balancing code were not fully open because of their coupling with internal systems.
The release exposes parts of a production-scale LLM inference stack, including scheduling, caching, execution and deployment patterns. DeepSeek said sharing the deployment experience could help developers and enterprises optimize inference throughput and latency and build private and public-cloud deployments more efficiently.
10月20日，DeepSeek宣布正式开源大规模云服务推理框架 DeepInfer。该框架已经用于 DeepSeek V4.1 Flash 的大规模线上推理服务。
据介绍，DeepInfer集成了高性能算子和深度优化的推理引擎，支持分布式推理、推测解码、SWA Bounded Replay、结构化输出等能力，并能够运行在 NVIDIA 与华为昇腾平台上。按照 DeepSeek 公布的数据，目前基于这套系统运行的单一集群服务吞吐已经达到每天155万亿 Token，按谷价计算，整体收入达到每卡每小时38美元。
DeepSeek同时开源了生产级 Token 编解码组件 deepseek-recipe，用于在主流大模型 API 协议和模型 Token 之间进行高效、准确转换，其处理逻辑与 DeepSeek 官方 API 完整对齐，从而使第三方部署能够尽可能获得与官方服务一致的 API 体验。
#DeepSeek#DeepInfer#DeepSeek-V4.1-Flash#deepseek-recipe#NVIDIA#Huawei Ascend#SM100 Blackwell#Ascend 950