Create

Sign in to ReadmeX

or

How we handle your data: Privacy Policy

ReadmeX

Your AI reading companion

What would you like to catch up on?

Pick a briefing, or ask your own question.

Your community and people briefings stay personal to you.

kimi-k3-in-c

kimi-k3-in-c• Pinned

kimi-k3-in-c: Kimi K3 inference in portable C99, with no BLAS, framework, or GPU required.

What problem it solves

Running the Kimi K3 model normally presents a storage and memory barrier because its checkpoint is very large. This project provides a portable C99 inference engine that streams model data so the complete model can run on a CPU without requiring GPU infrastructure or a deep-learning framework.

Key features

  • Portable C99 inference — The engine is implemented in portable C99 and does not depend on BLAS, a machine-learning framework, or a GPU. It builds for Linux, macOS, and Windows.

    Source: README, src

  • Streaming model data — The packed trunk and routed experts are streamed from storage rather than requiring the full checkpoint to remain resident. Memory presets provide adjustable budgets for the trunk and expert cache.

    Source: README, scripts

  • Exact validation gates — The test suite compares teacher forcing, greedy decoding, and incremental generation against reference fixtures. The repository also includes tools for replaying an expert cache trace.

    Source: README, tests, tools

  • Text chat mode — The engine can use the official Kimi K3 XTML format for text conversations, including system messages, JSONL history, greedy decoding, and optional sampling controls. It also provides an interactive REPL with session commands.

    Source: README, examples

  • Portable setup and tooling — The project includes Make and CMake workflows, environment diagnostics, resumable checkpoint downloading, trunk packing, and Python-based conformance and cache-replay tools.

    Source: README, Makefile, CMakeLists.txt

Quick start

Copied verbatim from the project's README.

git clone https://github.com/FareedKhan-dev/kimi-k3-in-c.git
cd kimi-k3-in-c

make -j            # seconds. Seven C files, a compiler and OpenMP
make test          # under a minute
./bin/k3 ~/k3model --trunk ~/k3trunk --preset workstation \
         --tok ~/k3model --prompt "The capital of France is" --gen 8 --incremental

Who it may suit

Inference: may suit developers interested in portable C99 systems, model streaming, or running Kimi K3 without a GPU or machine-learning framework.

Topics to discuss

  • Which additional platforms or CPU architectures should receive first-class kernel support?

  • How should the community validate streaming and incremental-generation behavior across different storage layouts?

  • What tooling would help developers inspect chat history, saved state, and cache traces more easily?

Written by ReadmeX AI from the project's README, source and public GitHub data; it may contain mistakes.

Read the original README https://github.com/FareedKhan-dev/kimi-k3-in-c#readme

ReadmeX opened this community for a trending project; it is not run by the project. Maintainers can claim it from the community page.

github.comhttps://github.com/FareedKhan-dev/kimi-k3-in-c

Comments

…