kimi-k3-in-c runs inference on a 2.78-trillion parameter mixture-of-experts language model using as little as 8.24 GB of RAM on a single CPU by streaming non-resident experts from disk. It is written in portable C99 with zero dependencies and produces byte-identical output to running the model fully resident.
Project overview
It makes a trillion-parameter model runnable on consumer-grade CPU hardware by keeping the dense trunk in memory and streaming sleeping routed experts from disk in their packed 4-bit form, achieving byte-identical output regardless of the memory preset chosen.
Deployment
Refer to project documentation
License
Apache-2.0
Best for
Developers and researchers who need to run a 2.78-trillion parameter mixture-of-experts model on a single CPU and want byte-identical output without requiring a GPU, BLAS, or external frameworks.
Key capabilities
Executes LLM inference using portable C99 without BLAS, frameworks, or GPUs, fitting a massive model into small RAM.
Keeps the always-on dense trunk in memory while streaming the sleeping routed experts directly from disk in their packed 4-bit form.
Provides laptop, desktop, workstation, server, and max presets to scale RAM requirements dynamically while producing identical output.
Limitations and risks
Inference is explicitly very slow on low hardware, for example approximately 32 seconds per token at 8 GB RAM.
The engine targets Linux only. The tokenizer and config reader are portable, but the main binary is not for macOS, Windows, or WSL.
A partial download of the checkpoint does not fail loudly but produces wrong tokens. Users should verify checkpoint integrity before running inference.
Getting started
Clone the repository, run k3-doctor.sh to check the machine, then build using make or CMake. Run 'make test' to verify the engine. Download the 1.56 TB checkpoint using download-model.sh, pack the trunk using pack-trunk.sh, and run the binary. Setup difficulty is medium because it requires downloading a 1.56 TB checkpoint.