Kimi K3 features 2.8 trillion parameters, effectively activating 16 out of 896 experts for each token processed, utilizing MXFP4 weights and MXFP8 activation values. The sparse expert architecture reduces the actual computational load, while low-precision weights compress storage requirements. However, it still cannot eliminate the memory pressure caused by the massive parameter scale. A single DGX B200 has a total of 1.44TB of memory, which theoretically only approaches the base volume of K3's four-bit weights. When accounting for caching, activation values, and runtime overhead, it is difficult to fully accommodate the model. The B300 and MI355X GPUs, each offering 288GB of memory, are more suitable for deploying ultra-large models. The bigger bottleneck lies in communication. While multiple B200 units can collectively hold the model, the experts distributed across different GPUs require frequent data exchanges, and the efficiency of cross-server transfers is significantly weaker compared to the high-speed NVLink within the GB300 NVL72. Therefore, Kimi officially recommends using ultra-nodes composed of 64 or more accelerators. High-density deployments also require supporting high-speed interconnects, parallel software stacks, as well as corresponding power supply and cooling systems. In summary: save on computation, consume memory, and prioritize interconnects.