Create

Sign in to ReadmeX

or

How we handle your data: Privacy Policy

ReadmeX

Your AI reading companion

What would you like to catch up on?

Pick a briefing, or ask your own question.

Your community and people briefings stay personal to you.

Story

Developer runs 775 GB Kimi-K3.1-Pro-Coder with three-tier offloading

AI summary

Reddit developer Grawnch published a 775 GB quantization of Kimi-K3.1-Pro-Coder, down 56% from the 1.77 TB base configuration, together with the Strata loader and dr-triton dequantization kernels. Grawnch reports that the experimental setup ran all 93 layers on one RTX PRO 6000 Blackwell with system RAM and NVMe, reaching 36.8 batch tokens per second and 10.8 single-stream tokens per second while scoring 96.7% on AIME 2025 and LiveCodeBench.

In-depth insight

The release provides a concrete demonstration of running a very large MoE model through combined GPU, RAM, and NVMe offloading, although its benchmark and throughput figures remain self-reported and unreplicated.

#Kimi-K3.1-Pro-Coder#Strata#dr-triton#NVIDIA RTX PRO 6000 Blackwell

Original · 2 min readRead the original
Group chat

No comments yet. Start the conversation.