Trillion parameter open source model runs 981 words/second, Cerebras tests Kimi K2.6 accelerates 29 times According to Beating monitoring, wafer level chip company Cerebras announced the launch of the trillion parameter large model Kimi K2.6 in enterprise testing. By distributing weights across multiple 12 inch silicon wafers and streaming activation values, it eliminates the interconnection delay of traditional board level communication. The third-party evaluation agency Artificial Analysis has tested that its generation speed reaches 981 tokens/s, which is 6.7 times faster than mainstream GPU cloud services. In a long text task with 10000 inputs and 500 output tokens, the total response time has been reduced from 163.7 seconds on the official Kimi interface to 5.6 seconds, a speed increase of 29 times. Due to the trillion parameter model exceeding the capacity limit of a single silicon wafer, the system assigns its weight to the flow transfer activation value in multiple wafers. Inside the wafer, the inter layer communication runs entirely on the on-chip network fabric, with a physical communication bandwidth that is over 200 times that of NVLink in Nvidia's NVL72 architecture. In conjunction with distributed computing optimization, Kimi K2.6 uses original 4-bit (4-bit) weights for low loss storage, 16 bit (16 bit) floating-point numbers to maintain accuracy during computation, and customized operator kernels and speculative decoding to achieve real-time operation. [Original link]