MXFP4

Headlines · 1

1. 6d
   kimi-k3-in-c runs Kimi K3 inference on one CPU with portable C99

   The open-source project kimi-k3-in-c implements Kimi K3 inference in portable C99 without BLAS, a framework or a GPU path, according to its documentation. Its author reports that the 2.78-trillion-parameter model produced the same byte-identical output while running with 8 GB, 32 GB, 64 GB or 128 GB-plus of RAM. Measured performance ranged from 26.5 seconds per token with 8 GB to 5.6 seconds with 128 GB-plus, when the model fit entirely in memory. All figures came from one dual-socket AMD EPYC 7763 workstation with 124 cores and 228 GB of RAM. The project reports a 1.56 TB checkpoint and 8.24 GB peak RSS for the 8 GB configuration. Its macOS, Windows and NEON ports, chat mode, ultra preset, faster MXFP4 kernels and parser fixes were contributed by other participants.

   GitHub Trending（每日） · 🔥 0

Experience and discussion from the community

Share my MXFP4 experienceAsk about MXFP4