FareedKhan-dev

Headlines · 1

1. 6d
   kimi-k3-in-c runs Kimi K3 inference on one CPU with portable C99

   The open-source project kimi-k3-in-c implements Kimi K3 inference in portable C99 without BLAS, a framework or a GPU path, according to its documentation. Its author reports that the 2.78-trillion-parameter model produced the same byte-identical output while running with 8 GB, 32 GB, 64 GB or 128 GB-plus of RAM. Measured performance ranged from 26.5 seconds per token with 8 GB to 5.6 seconds with 128 GB-plus, when the model fit entirely in memory. All figures came from one dual-socket AMD EPYC 7763 workstation with 124 cores and 228 GB of RAM. The project reports a 1.56 TB checkpoint and 8.24 GB peak RSS for the 8 GB configuration. Its macOS, Windows and NEON ports, chat mode, ultra preset, faster MXFP4 kernels and parser fixes were contributed by other participants.

   GitHub Trending（每日） · 🔥 0

Experience and discussion from the community

Share my FareedKhan-dev experienceAsk about FareedKhan-dev

All

kimi-k3-in-c•4d Pinned

kimi-k3-in-c: Kimi K3 inference in portable C99, with no BLAS, framework, or GPU required.

What problem it solves Running the Kimi K3 model normally presents a storage and memory barrier because its checkpoint is very large. This project provides a portable C99 inference engine that streams model data so the complete model can run on a CPU without requiring GPU infrastructure or a deep-learning framework. Key features Portable C99 inference — The engine is implemented in portable C99 an…

github.comhttps://github.com/FareedKhan-dev/kimi-k3-in-c