Cerebras Deploys Trillion-Parameter Models on Wafer-Scale Chips

Cerebras has deployed trillion-parameter AI models using its WSE-3T wafer-scale chips. The company split large frontier models across multiple wafers to run multi-trillion parameter models. This deployment approach achieves speeds of 1000 to 2000 plus tokens per second.

A single Cerebras WSE-3T wafer contains 44 GB of on-chip memory. Cerebras used the open source model Kimi K3 as an illustrative example for deployment. Kimi K3 is an open source model with approximately 3 trillion parameters. The model scores 88.3 percent on Terminal-Bench 2.1 and 91.2 percent on BrowseComp. Cerebras requires a total of 59 wafers to support the Kimi K3 model.

Cerebras executes models in a pipeline where activations are passed between wafers via RDMA over Ethernet. The latency for passing activations between wafers is roughly 2 microseconds per hop. Model weights typically stay on the chip and do not need to move between wafers. Cerebras serves multiple requests simultaneously by keeping the pipeline full with different requests at each stage. The company uses micro-batches to process a small group of requests rather than a single request per stage.

Cerebras offers a free API for deploying models on its inference cloud.