Xolotl engine runs 552B DeepSeek-V4.1-Flash on a single RTX 4090, developer reports
Reddit user indrasmirror released Xolotl, a Strata-based inference engine, claiming it runs the full 552B DeepSeek-V4.1-Flash on one gaming PC: a single RTX 4090 (24GB), a Ryzen 7 5700X, 90GB of DDR4 and an NVMe drive, with weights spread across VRAM, pinned RAM and NVMe and only the experts each token needs pulled in. The developer self-reports about 27 tok/s decode, 180–320 tok/s prefill, 512K context, and 21/21 HumanEval, 10/10 tool calls and 3/3 agentic bug-fix loops through the real server with thinking on; none of these figures has been independently verified. The post also says the same engine runs GLM-5.3-Flash, Qwen3.8-Flash-Next and Qwen3.8-27B, that a 152 GiB uncensored V4.1 quant is on Hugging Face, and that the 510GB DeepSeek-V4.1-Flash-1017 update will be abliterated and requantized within a couple of days.
The post claims a 552B mixture-of-experts model can run on a single 24GB consumer GPU by spreading weights across VRAM, pinned RAM and NVMe and loading only the experts each token needs.
Hey all, been meaning to post this for a while.
I've been building an inference engine based off Strata called Xolotl that runs DeepSeek-V4.1-Flash, all 552B of it, on one gaming PC. My setup is a single RTX 4090 (24GB), a Ryzen 7 5700X, 90GB of DDR4 and an NVMe drive, nothing fancy. The weights get spread across VRAM, pinned RAM and NVMe, and the engine pulls in the experts each token actually needs.
What I'm getting on that machine:
- 27 tok/s decode, 180-320 tok/s prefill
- 512K context
- 21/21 HumanEval, 10/10 tool calls, 3/3 agentic bug-fix loops, all through the real server with thinking on
It also runs GLM-5.3-Flash, Qwen3.8-Flash-Next and Qwen3.8-27B from the same engine.
#Xolotl#Strata#DeepSeek-V4.1-Flash#GLM-5.3-Flash#Qwen3.8-Flash-Next#Qwen3.8-27B#Hugging Face#Nvidia RTX 4090