kimi-k3-in-c runs Kimi K3 inference on one CPU with portable C99
The open-source project kimi-k3-in-c implements Kimi K3 inference in portable C99 without BLAS, a framework or a GPU path, according to its documentation. Its author reports that the 2.78-trillion-parameter model produced the same byte-identical output while running with 8 GB, 32 GB, 64 GB or 128 GB-plus of RAM. Measured performance ranged from 26.5 seconds per token with 8 GB to 5.6 seconds with 128 GB-plus, when the model fit entirely in memory. All figures came from one dual-socket AMD EPYC 7763 workstation with 124 cores and 228 GB of RAM. The project reports a 1.56 TB checkpoint and 8.24 GB peak RSS for the 8 GB configuration. Its macOS, Windows and NEON ports, chat mode, ultra preset, faster MXFP4 kernels and parser fixes were contributed by other participants.
The project documents a CPU-only approach that streams most of Kimi K3’s routed experts from storage rather than keeping them in memory. It reports byte-identical output across memory configurations while using no GPU path, BLAS library or inference framework.
A 2.78-trillion-parameter Kimi K3 running inference on a single CPU in 8.24 GB of RAM. Portable C99: no BLAS, no framework, no GPU.
kimi-k3-in-c
A 2.78-trillion-parameter model. One CPU. 8 GB of RAM.
Kimi K3 inference in portable C99. No BLAS. No framework. No GPU.
Some source content is omitted. Continue reading on the original site.