OpenAI's Cerebras chip is pretty interesting 1. Wafer-Scale Size: The world's largest chip—how big is it? About the size of a dinner plate. Transistor count: It has 4 trillion transistors (for comparison, H100 only has 80 billion). 2. Ultra-high On-chip Memory Bandwidth (On-chip SRAM) This is Cerebras' key weapon for outperforming GPUs. Eliminating bottlenecks: In traditional GPU architectures, model computation requires frequent data transfers between memory (HBM) and compute cores, which leads to significant energy consumption and latency. (Memory computation may dilute HBM growth rates, but the market is big enough, and SRAM costs are high, so it doesn’t pose a major threat to Samsung, SK Hynix, or Micron in the short term.) Full on-chip storage: Cerebras has up to 44GB of on-chip SRAM memory with bandwidth reaching 21 PB (PetaBytes) per second. This means most of the model’s weights can be fully stored within the chip, with read/write speeds thousands of times faster than GPU memory, enabling “instant” inference like OpenAI models. 3. Simplified Programming and Scaling Single machine = cluster: Since the chip itself is large enough, one Cerebras node (CS-3) has computing power equivalent to dozens or even hundreds of traditional GPU nodes. No need to split models: Developers don’t need to break a large model into multiple parts or deal with complex cross-server communication (Model Parallelism) like on GPU clusters. For Cerebras, the entire model runs on “one” chip. 4. Sparse Optimization for Large Language Models (LLM) Handling zero values: AI models have many weights that are “zero” (sparsity), and traditional GPUs still perform unnecessary calculations on these zeros. Cerebras chips have a built-in sparse computation engine that can skip zero values directly, further squeezing out performance.