Xolotl引擎让552B DeepSeek跑在单卡4090
r/LocalLLaMA 用户 indrasmirror 发布了基于 Strata 的推理引擎 Xolotl，称可在单台游戏 PC 上运行完整的 552B DeepSeek-V4.1-Flash：硬件为单张 RTX 4090（24GB）、Ryzen 7 5700X、90GB DDR4 和一块 NVMe 硬盘，权重分布在显存、锁定内存与 NVMe 之间，并按 token 只加载所需的专家。该开发者自报解码约 27 tok/s、预填充 180–320 tok/s、512K 上下文，以及 21/21 HumanEval、10/10 工具调用和 3/3 智能体修 bug 循环（开启 thinking、走真实服务器），这些数字均未经独立验证。帖中还称同一引擎可运行 GLM-5.3-Flash、Qwen3.8-Flash-Next 和 Qwen3.8-27B，并已在 Hugging Face 放出 152 GiB 的“无审查”V4.1 量化版本，计划在几天内对 510GB 的 DeepSeek-V4.1-Flash-1017 重新做消融与量化。
该帖称一个 552B 的 MoE 模型可在单张 24GB 消费级显卡上运行，办法是把权重分散到显存、锁定内存和 NVMe，并且每个 token 只加载所需的专家。
Hey all, been meaning to post this for a while.
I've been building an inference engine based off Strata called Xolotl that runs DeepSeek-V4.1-Flash, all 552B of it, on one gaming PC. My setup is a single RTX 4090 (24GB), a Ryzen 7 5700X, 90GB of DDR4 and an NVMe drive, nothing fancy. The weights get spread across VRAM, pinned RAM and NVMe, and the engine pulls in the experts each token actually needs.
What I'm getting on that machine:
- 27 tok/s decode, 180-320 tok/s prefill
- 512K context
- 21/21 HumanEval, 10/10 tool calls, 3/3 agentic bug-fix loops, all through the real server with thinking on
It also runs GLM-5.3-Flash, Qwen3.8-Flash-Next and Qwen3.8-27B from the same engine.
#Xolotl#Strata#DeepSeek-V4.1-Flash#GLM-5.3-Flash#Qwen3.8-Flash-Next#Qwen3.8-27B#Hugging Face#Nvidia RTX 4090