Cerebras Details Multi-Wafer Inference for Trillion-Parameter Models
Cerebras has described how it maps the open-weight Kimi K3 model across 59 WSE-3T wafers, using 4-bit weights, an 8-bit KV cache, and a pipelined runtime. The company says activations move between wafers over RDMA Ethernet while multiple requests and micro-batches occupy different pipeline stages. Cerebras claims the multi-wafer system can run multi-trillion-parameter models at 1,000–2,000+ tokens per second, but no independent measurement was supplied.
The account provides a concrete deployment design for fitting and serving a roughly 3-trillion-parameter model across wafer-scale hardware, while its throughput claim remains unverified.
One of the most frequently asked questions we get is how we deploy large models on Cerebras wafer-scale chips. A single Cerebras WSE-3T wafer has 44 GB of on-chip memory, and trillion-parameter frontier models need terabytes for their weights alone.
In this post, we’ll explain how we split a large frontier model across multiple WSE-3T wafers, to run multi-trillion parameter models at 1000-2000+ tokens per second.
We’ll use the open source model Kimi K3, with ~3 trillion parameters, as an illustrative example.
By the end, you'll understand:
- How we fit the full model into on-chip memory
- How we map the model across multiple chips
- How we handle a high # of concurrent requests
Breaking it down with Kimi K3
Kimi K3 is an open source, 3T parameter model with 69 linear-attention layers and 24 dense layers.