Cerebras has laid out how it runs trillion-parameter AI models on its wafer-scale chips, using the open-source Kimi K3 as a worked example. The company says the method lets it run multi-trillion-parameter models at 1,000 to 2,000-plus tokens per second by splitting each model across several chips.
The problem starts with memory. Each Cerebras WSE-3T wafer carries 44 GB of on-chip memory, while a trillion-parameter model needs terabytes just to store its weights. Kimi K3 has about 3 trillion parameters, arranged in 69 linear-attention layers and 24 dense layers. Cerebras calculates that each layer needs roughly 16 GB when the weights are stored at 4 bits.
How many layers fit on a wafer depends on the layer type. Dense layers, also called full-attention layers, store keys and values for every token in every active request. That cache takes up room, so Cerebras places one dense layer per wafer and uses the leftover memory for the cache. Linear-attention layers work differently. They keep a small, fixed-size state for each request no matter how long the context gets, so two of them fit on one wafer, using 32 GB.
Run through those numbers and the layout comes to 35 wafers for the linear layers and 24 for the dense layers, or 59 wafers in total.
The wafers then have to work together. Cerebras says its compiler maps the model's layers across regions of wafers, and its runtime sets the path tokens take between them. Only activations move from one wafer to the next, and the company says that transfer runs over RDMA on Ethernet at roughly 2 microseconds per hop. Because most layers fit entirely on a single chip, the model weights usually stay where they are. Only the intermediate results travel.
Cerebras compares the setup to an assembly line. In the company's example, Wafer 1 runs layers 1 and 2 and sends its output to Wafer 2, which runs layers 3 and 4 and passes results along, until the last wafer produces the next token. A single request moves through the line one stage at a time.
That raises an obvious question. If each token has to wait its turn at every wafer, does most of the hardware sit idle? Cerebras says no, because it does not process one request at a time. It keeps the pipeline full, so while one stage works on a token for one request, the next stage works on a token for a different request. In production, the company uses much finer-grained stages than the layer-by-layer example, and each stage can process a micro-batch, a small group of requests, rather than a single one.
The tradeoff is familiar to anyone who has tuned a serving system. Smaller micro-batches make each individual request faster. Larger micro-batches push up total throughput. The number of requests in flight at any moment is roughly the number of stages multiplied by the micro-batch size.
The amount of memory left over for the KV cache sets the ceiling on concurrent requests. Every active request needs its own cache on each dense-layer wafer, so the memory on those chips limits how many can run at once. Cerebras calculates the cache requirement for a 256k-token average request, using 4-bit weight quantization and an 8-bit KV cache for the example.
Cerebras also argues that concurrency is not the full picture. Requests on its system do not wait for each other to finish the way a GPU batch does. They stream through the pipeline continuously. The company says the more useful measure of an inference system is total token throughput per unit of power, such as tokens per second per megawatt, measured at the same output speed.
On performance, Cerebras points to Kimi K3's benchmark results. The model scores 88.3% on Terminal-Bench 2.1, compared with 88.8% for GPT-5.6 Sol. On BrowseComp, it scores 91.2%, compared with 90.4% for GPT-5.6 Sol. The company says those results show frontier-level coding and agentic capabilities, and that the open model outperforms leading proprietary models on some benchmarks.
Cerebras says the same approach applies to larger models. Engineers calculate the memory needed for the weights and request state, spread the model across wafers, and connect them into a pipeline. Each added wafer brings more compute and more on-chip memory, which the company says lets it handle models larger than any single chip can hold.
This article was written with the assistance of AI.
News Factory APP - agentic news to boost your SEO & AEO.