“Former Google TPU Architect: The Real Bottleneck of AI is Not Computing Power”

In this two-hour interview, Reiner Pope gradually laid out the physics behind training and inference on the blackboard. His insights are crucial for understanding the AI industry chain—especially concerning chips, memory, and interconnected devices.

However, the original text is very complex, and ordinary readers may find it exhausting.

So, without altering any of Reiner’s original meaning, I will do two things:

First, rephrase the content in simple terms.

Second, extract the key points from an investment perspective.

The article is divided into three sections: what the current situation is, what the underlying principles are, and which industries will be affected in the future.

1. First, clarify it in one sentence

The core judgment of Reiner's lecture is: the real bottleneck of AI is not computing power, but the speed of data transfer. This bottleneck has no solution in the short term.

If you remember only one thing, let it be this sentence. Almost all industrial implications stem from here.

Why is this important? Because where the money flows in the entire AI industry chain, who benefits and who suffers, depends on “where the bottleneck is.” If the bottleneck is computing power, then GPU manufacturers are the absolute winners; if the bottleneck is data transfer, then the money will go to another set of companies—HBM memory, interconnects between racks, cables, switches, liquid cooling, power supplies.

Reiner's answer is very clear: the bottleneck is the latter. This is something he can directly observe from the capital expenditure structure of large companies—according to industry estimates, they are spending about half of their budget on memory this year.

2. Computing power is sufficient, what’s lacking is “the mover”

To understand why computing power is not lacking while memory is, let’s use an analogy.

Imagine a GPU as a super-efficient accountant. Given a stack of ledgers (model parameters), he can calculate quickly. The problem is: the ledgers are not at his fingertips; they are in a warehouse. Each time he needs to calculate, someone has to bring the ledgers from the warehouse to his desk, and after he finishes, they must be returned.

There are two times involved:

Calculation time: how fast can calculations be performed

Transfer time: how slow is the movement of the ledgers

As per usual, the article is lengthy; you can head directly to the public account.

https://mp.weixin.qq.com/s/qZTnUHKBHkEYX2GuVPFKQw

免责声明：本文章仅代表作者个人观点，不代表本平台的立场和观点。本文章仅供信息分享，不构成对任何人的任何投资建议。用户与作者之间的任何争议，与本平台无关。如网页中刊载的文章或图片涉及侵权，请提供相关的权利证明和身份证明发送邮件到support@aicoin.com，本平台相关工作人员将会进行核查。