Zhī pǔ goes live today with GLM-5.3-FlashX, with inference speed up to 200 tokens/s, which is 5 times faster and 2.5 times more expensive than the GLM-5.3-Flash model, API open synchronously, model name: GLM-5.3-FlashX

Behind this is based on the inference computing power provided by 100,000 domestic chips, further increased investment in Infra side and inference optimization

GLM's Infra, built by GLM

The Infra running Flash is built by the GLM-5.3 driven Agent, achieving production launch in less than two weeks from model adaptation, with end-to-end throughput reaching an initial baseline of 3 times

Performance evolution of GLM-5.3-Flash from first run to official launch

During the construction process, humans are responsible for setting goals, defining boundaries, building feedback environments, and key modification reviews, while testing, logging, Trace, and microbenchmarks are all handed to the Agent, which autonomously checks issues, modifies code, runs experiments, verifies locally, and then integrates back into the complete service for acceptance, completing "dense feedback"

Infra Agent optimization closed-loop based on dense feedback

Below are three cases from building and optimizing this inference system.

Case 1: Input is FP32, but calculation used TF32

The KDA's CP path had accuracy deviations, especially evident in long contexts. The Agent traced the state merge and update process and found the issue in two places related to tl.dot: input was FP32, but calculation defaulted to TF32, causing the error to accumulate over time.

Both of these were explicitly added with input_precision="tf32x3". The fix has been merged into the upstream of Flash Linear Attention.

Case 2: KV Transfer was blocked by GIL

With the same workload, the performance gap after Prefill and transfer was required to be no more than 5%, but testing showed over 20%. The Agent reviewed the Trace and found that the Python execution of KV Transfer could not overlap with DeepEP’s dispatch/combine calls.

Further investigation revealed that two C++ calls in DeepEP v1.2.1 did not release the GIL, causing the Python thread responsible for Mooncake Transfer to be unable to acquire the lock, delaying the transfer task submission. After modification, the performance gap under the same testing conditions dropped to within 1%.

Investigation and fixing of KV Transfer concurrency bottleneck

Case 3: Same group of calculations done four times

KDA Decode divided along the V dimension, with the same group of FP32 normalization and gated calculations repeatedly done four times due to different block divisions.

The Agent merged these blocks into the same thread block to compute in batches ahead of time, sharing intermediate results. This reduced parallelism a bit and eliminated redundant calculations; this step increased operator performance to 1.71 times of the pre-optimization level.

Performance evolution of KDA Decode operator from basic implementation to production version

免责声明：本文章仅代表作者个人观点，不代表本平台的立场和观点。本文章仅供信息分享，不构成对任何人的任何投资建议。用户与作者之间的任何争议，与本平台无关。如网页中刊载的文章或图片涉及侵权，请提供相关的权利证明和身份证明发送邮件到support@aicoin.com，本平台相关工作人员将会进行核查。