Developer runs 775 GB Kimi-K3.1-Pro-Coder with three-tier offloading
AI summary
Reddit developer Grawnch published a 775 GB quantization of Kimi-K3.1-Pro-Coder, down 56% from the 1.77 TB base configuration, together with the Strata loader and dr-triton dequantization kernels. Grawnch reports that the experimental setup ran all 93 layers on one RTX PRO 6000 Blackwell with system RAM and NVMe, reaching 36.8 batch tokens per second and 10.8 single-stream tokens per second while scoring 96.7% on AIME 2025 and LiveCodeBench.
In-depth insight
The release provides a concrete demonstration of running a very large MoE model through combined GPU, RAM, and NVMe offloading, although its benchmark and throughput figures remain self-reported and unreplicated.