Homelab Runs Kimi K3 on CPU With Second-Gen Optane
A Reddit homelab builder reportedly ran Moonshot AI’s 2.8-trillion-parameter Kimi K3 on a 40-core third-generation Intel Xeon Scalable CPU using discontinued second-generation Optane PMem 200 modules, without a GPU for inference. Before thread-count or NUMA-affinity tuning, the system generated 0.85 tokens per second with 1-bit quantization and 0.75 tokens per second with 3-bit quantization. The builder said the Optane modules became extremely hot during sustained inference and advised treating active DIMM cooling as mandatory.
The experiment shows that CPU-only inference of a very large model is technically possible with discontinued persistent memory. However, the very low pre-tuning speed, substantial cooling requirements and reliance on one builder’s account sharply limit its practical value.
A homelab builder on Reddit has run Moonshot AI’s 2.8-trillion-parameter Kimi K3 AI model on discontinued second-gen Optane PMem 200 memory, using a 40-core 3rd-gen Xeon Scalable CPU and not a GPU for inference. The build runs hot, and the builder’s advice is that you “treat active DIMM cooling as mandatory,” with dual 12W blower fans aimed at the modules to get temperatures under control. The project, a follow-up to a separate user’s first-gen Optane build earlier this year, runs at a sluggish 0.85 tokens per second even with 1-bit quantization.
Second-gen Optane PMem server running Kimi K3 (2.8T params) on CPU only, plus a critical thermal warning that could save your RAM from r/homelab
Kimi K3 is absolutely not built for home use, in our estimation, with Moonshot AI recommending data center inference engines to run it, and even the smallest community quantization needs at least 610GB of RAM. Using secondhand Optane, instead, is one of the few ways to reach that capacity at home. It doesn’t help that DRAM prices have skyrocketed, and the builder says the Optane memory costs “a fraction” of the equivalent.
#Kimi K3#Moonshot AI#Intel Optane Persistent Memory 200#Intel Xeon Scalable