Art of Speculation
Art of Speculation|6月 27, 2026 19:22
In the latest episode of All In Podcast, what do several experts think about Micron, storage bottlenecks, Chinese open source models, and distributed inference This All In Podcast has a lot of information, so I have selected a few topics that I think are most worth sharing and organized them. First, let's talk about the development of open source models in China, which is faster than expected Zhipu AI has released a new generation of Frontier level open source model GLM with 52744 million parameters and 1 million token context, completely following the MIT open source protocol. The evaluation data is quite astonishing: it defeated GPT-5.5 in software engineering programming benchmark tests, and is less than 1% behind Anthropic's top of the line Claude Opus 4.8, but the API price is 85% cheaper than the same performance American model. There is an interesting detail in the program about a method used by Chinese teams to accelerate catch-up: using thousands of mobile phones and iPads to form a device farm, using encrypted accounts to ask high-density questions to the API of top tier Frontier models in the United States, harvesting the other party's inference links, and feeding them to their own open-source models for reinforcement training. This is equivalent to treating the standard answers that American laboratories spend huge amounts of money to produce as a cheat sheet, achieving similar performance at a very low cost. Sacks' attitude towards this is quite sharp. He criticized Anthropic's Dario for pushing the US government to establish a cumbersome security approval process, which actually slowed down the pace of the US itself. The Fable model was forced to be taken down due to jailbreak allegations, and OpenAI's new model approval is also facing difficulties. His judgment is that the Chinese model is currently about 9 months behind in technology and 24 months behind in chips, but it has already completed the training of the GLM5 family using local chips such as Huawei Ascend. In the future, these "AI boxes" optimized for local chips and cheap and easy to use are likely to be sold at low prices to the global market, while the United States itself is setting various restrictions, which has actually driven out this trillion dollar export market. Micron's financial report this time provided a precise positioning in the program: DRAM is the real bottleneck of the entire AI wave Micron's revenue in this quarter surged fourfold year-on-year, from 9 billion to 42 billion, far exceeding expectations. The HBM production capacity for 2026 has long been sold out. In the program, there is a viewpoint that is quite straightforward: Previously, people searched for various Japanese small auxiliary material companies on Twitter as "bottleneck stocks", but the real lifeline is only DRAM, especially HBM. The reason is simple: the bandwidth and capacity of memory determine the physical ceiling of inference performance for all large models, which is a hard constraint that cannot be bypassed. Even mentioning the super factory that Musk is building, the core technology is aimed at DRAM, not fiber optic, power supply, or NAND flash memory. Micron has also made an interesting change in its business model this time: signing long-term supply agreements with core cloud providers with "price floor and ceiling" protection, locking in 50% of future revenue. This means that even if the industry cycle declines in the future, the minimum contract protection price will still be higher than the gross profit peak of any previous cycle. Entering the barrier area, although China's Changxin Storage is preparing to go public and may use low-priced mid to low end consumer grade memory to alleviate the cost pressure of large companies such as Apple in the future, only Micron, SK Hynix, and Samsung can produce the top-level HBM required for AI servers globally, with extremely high technological difficulty that cannot be caught up in the short term. A rather exaggerated prediction was made in the program: next year, 30% to 40% of the global large-scale capital expenditure will directly flow to DRAM chip manufacturers. This cost surge has led Apple to raise the retail prices of MacBook and Mac Studio across the board. Edge computing and distributed reasoning are the most imaginative content in this issue, and I would like to share some interesting ideas Tesla applied for a hardware trademark called "Megapod" on June 18th. The physical logic behind it is that building a 1-gigawatt data center on the ground requires an extremely lengthy approval process for land, energy consumption, and liquid cooling. The idea of Megapod is to integrate GPUs, battery networks, and cooling systems into a containerized modular data center, which can be directly deployed in the Tesla Supercharging Station network that has already been approved, has an existing power grid, and idle land, bypassing the biggest bottleneck of traditional data center construction - approval and power access. The logic of the distributed inference line is also quite interesting: the model can answer questions in two stages, the Prefill stage for understanding the problem and the Decoding stage for high bandwidth and high memory consumption. Large funds can purchase old graphics cards that have been depreciated, and front-end plug-in chips specifically designed for decoding optimization can form a lower cost distributed inference network. A crazier idea is to offer discounts to users who purchase household energy storage batteries Powerwall in the future, forcing the installation of AI chips in each battery, coupled with Starlink satellite connections. When the battery is idle, it will automatically form a huge distributed P2P inference pool and obtain a continuous supply of almost free offshore computing power. If this idea is really realized, it will be a dimensionality reduction blow to traditional cloud giants. The craziest part is space computing power. Building a 1-gigawatt data center on the ground requires $35 billion in chip costs and $25 billion in cooling labor costs, and also faces various land disputes. But as SpaceX's starships become fully reusable, the cost of launching 1 gigawatt of computing power into orbit through laser interconnect could plummet to just $5 billion. The natural cold environment and almost infinite solar energy in space may make the operational economics of space data centers surpass those of surface data centers within 3 to 4 years.
+4
Mentioned
Share To

Timeline

HotFlash

APP

X

Telegram

Facebook

Reddit

CopyLink

Hot Reads