家庭实验机用第二代Optane纯CPU运行Kimi K3
一名Reddit家庭实验室建造者据称使用已停产的第二代Optane PMem 200模块和40核第三代Intel Xeon Scalable CPU，在不使用GPU进行推理的情况下运行了Moonshot AI的2.8万亿参数Kimi K3。在调整线程数或NUMA亲和性前，系统在1-bit量化下生成速度为每秒0.85个token，在3-bit量化下为每秒0.75个token。建造者称，Optane模块在持续推理期间会变得极热，并建议将主动DIMM散热视为必要条件。
该实验表明，利用已停产的持久内存纯CPU运行超大模型在技术上可行。然而，预调优速度极低、散热负担较重，而且证据仅来自一名建造者的陈述，因此其实际价值受到明显限制。
A homelab builder on Reddit has run Moonshot AI’s 2.8-trillion-parameter Kimi K3 AI model on discontinued second-gen Optane PMem 200 memory, using a 40-core 3rd-gen Xeon Scalable CPU and not a GPU for inference. The build runs hot, and the builder’s advice is that you “treat active DIMM cooling as mandatory,” with dual 12W blower fans aimed at the modules to get temperatures under control. The project, a follow-up to a separate user’s first-gen Optane build earlier this year, runs at a sluggish 0.85 tokens per second even with 1-bit quantization.
Second-gen Optane PMem server running Kimi K3 (2.8T params) on CPU only, plus a critical thermal warning that could save your RAM from r/homelab
Kimi K3 is absolutely not built for home use, in our estimation, with Moonshot AI recommending data center inference engines to run it, and even the smallest community quantization needs at least 610GB of RAM. Using secondhand Optane, instead, is one of the few ways to reach that capacity at home. It doesn’t help that DRAM prices have skyrocketed, and the builder says the Optane memory costs “a fraction” of the equivalent.
#Kimi K3#Moonshot AI#Intel Optane Persistent Memory 200#Intel Xeon Scalable