Inside Cerebras WSE: a wafer-scale chip built to break the GPU memory wall
A long technical analysis walks through Cerebras's Wafer Scale Engine: instead of dicing a wafer, it turns a full 300mm wafer into a 21.5cm x 21.5cm chip with 84 dies, roughly 900,000 active cores, 4 trillion transistors and about 44GB of on-chip SRAM, with claimed on-chip bandwidth of 21PB/s. The piece argues that keeping weights in SRAM next to the compute units sidesteps the HBM memory wall that leaves GPUs idle while shuffling weights for every token. It also details reticle stitching across scribe lines, whole-wafer testing and fault-tolerant routing, and a four-layer “engine block” assembly of cold plate, wafer, flexible connector and PCB (23kW peak power, vertical power delivery, microchannel liquid cooling).
If wafer-scale silicon really delivers on low-latency inference, it offers a fundamentally different path in a market now dominated by GPUs.