Cerebras详解万亿参数模型多晶圆推理
Cerebras介绍了如何将开放权重模型Kimi K3映射到59块WSE-3T晶圆，采用4位权重、8位KV缓存和流水线运行时。模型激活值通过RDMA以太网在晶圆间传递，不同流水线和微批可同时处理多个请求。Cerebras称该系统可实现每秒1,000–2,000+个token，但目前没有独立测量予以证实。
该方案具体展示了如何在晶圆级硬件上部署约3万亿参数模型，但其吞吐性能尚未得到独立验证。
One of the most frequently asked questions we get is how we deploy large models on Cerebras wafer-scale chips. A single Cerebras WSE-3T wafer has 44 GB of on-chip memory, and trillion-parameter frontier models need terabytes for their weights alone.
In this post, we’ll explain how we split a large frontier model across multiple WSE-3T wafers, to run multi-trillion parameter models at 1000-2000+ tokens per second.
We’ll use the open source model Kimi K3, with ~3 trillion parameters, as an illustrative example.
By the end, you'll understand:
- How we fit the full model into on-chip memory
- How we map the model across multiple chips
- How we handle a high # of concurrent requests
Breaking it down with Kimi K3
Kimi K3 is an open source, 3T parameter model with 69 linear-attention layers and 24 dense layers.