How much memory is needed exactly? Translation and summary of this X post (long article) zeitgeist_labs Core argument (one sentence summary): Even though memory stocks such as Micron (MU) have surged from low levels, the demand for HBM's high bandwidth memory may still far exceed supply, especially with the explosive growth of workloads from "agentic AI". Memory stocks still have huge potential for further growth (the author believes it can last for another 10x). Don't be limited by historical highs or psychological factors, but should conduct a quantitative analysis of the underlying supply and demand. 1. Initial Example If I had bought $50000 worth of Micron stock in September last year, it would be worth approximately $489000 today. The author points out that people who haven't bought may feel "too late"; People who have already made a purchase may worry every day about whether they are a potential buyer. But this article uses a bottom-up analysis to argue that memory demand far exceeds current global production capacity. Why does AI need so much memory? (Technical foundation) LLM inference process: message → tokenization → GPU cluster performs massive matrix operations → generates replies. The key memory consumption comes from KV Cache (Key Value Cache): it grows linearly with the length of the context, and each concurrent session requires independent caching. GPU HBM capacity is limited (non scalable): ◦H100：80GB ◦H200：141GB ◦B200：192GB The upcoming B300:288GB Calculation example (taking Llama 3.1-70B real model as an example): Each token consumes approximately 160KB of memory. A single 128K context session requires approximately 20GB of memory (for only one user). The cutting-edge model (with a larger attention dimension) may require 40-100GB for a single 128K request. 3. Not all sessions are the same - 'proxy' workloads are key Simple queries (such as asking about weather): The memory requirement is extremely low, and it can serve thousands of devices simultaneously. • Medium complexity (such as debugging code): approximately 800MB. Complex tasks (such as lawyers uploading 50 page contract redlines): nearly 5GB. Agenetic session: self looping, multi tool calling, rapidly expanding context to over 100K+tokens. This is the direction of future mainstream (repeatedly mentioned by Sam Altman and Jensen Huang). Real example (proxy session diagram referenced in the post): A proxy workflow for enterprise customer churn analysis and win back strategy (AE/CSM may run multiple times in daily life): Firstly, perform SQL queries on the 25 lost enterprise accounts in Q2, with a total ARR of 6.9 million US dollars. The in-depth research on spawn sub agents supports tickets, product releases, public signals, and competitor evidence. Cross check product release records, support search, and CRM data. • Generate personalized win back email drafts+human approval process. Result: The context quickly approached the limit of 128K tokens (actually using 128050 tokens), retaining a large number of references, notes, tables, email drafts, etc. one This indicates that even relatively simple business workflows can quickly consume massive amounts of memory. How terrifying is the scale of global demand? Approximately 250 million knowledge workers worldwide. If each knowledge worker runs 10 concurrent proxy sessions with 100K tokens per day, the peak memory requirement for a single person is approximately 152GB. After scaling, memory demand shows explosive growth (not linear). The author estimates that in a higher adoption scenario, the global demand for HBM memory is approximately 60 times the expected production in 2026 (for example, a scenario requires 385 billion GB). 5. Current HBM supply situation (estimated by the author) The author cross validated two methods (wafer production capacity → GB production+revenue/ASP extrapolation) and referred to TrendForce, Counterpoint, company financial reports, etc. to obtain HBM production capacity estimates for Samsung, SK Hynix, and Micron. Conclusion: The global HBM production in 2026 is far from sufficient, and the three major manufacturers are currently in a "sold out" state, locking in prices and quantities many years ago. Can algorithm optimization save the situation? Yes, but not enough: Grouped Query Attention (GQA), Multi Latent Attention (MLA, such as DeepSeek), etc. can reduce KV Cache by 4-8 times or even more. In the next few years, it may be optimized to 10-16 times. But the demand side growth is on the order of magnitude (100x level): proxies replace chat, context shifts from 128K to millions of tokens, knowledge workers go from 0 to mass use. Optimization is a 'constant factor', demand is an 'exponential level', and the gap is still huge. 7. Final conclusion (author's viewpoint) We need a massive amount of memory. Financial analysts often make decisions based on historical performance or ATH (historical highs), but it is difficult to imagine the world where LLM is deeply embedded in daily work. In this world, memory is the core infrastructure, and companies producing HBM will receive unprecedented revenue. The author encourages readers to plug in different hypotheses (adoption rate, optimization speed, context length) for order of magnitude calculations. Appendix: The author provides a detailed explanation of the production capacity estimation methodology (wafer production model+revenue cross validation) and lists all reference sources (Micron financial report, SK Hynix, TrendForce, McKinsey, etc.). My brief review (as a supplement to the summary) This is a typical long article with bottom-up and order of magnitude thinking, with solid core logic (KV Cache+proxy concurrency+global knowledge worker scale). HBM does indeed have real supply bottlenecks (advanced packaging, yield, slow capacity ramp up), which is an industry consensus. However, the final outcome still depends on: Actual landing speed and ROI of AI agents • Algorithm/architecture optimization range Economic environment and capital expenditure The post itself leans heavily towards memory requirements and is suitable as a framework for thinking rather than direct investment advice. Do I need to further elaborate on a calculation, translate specific paragraphs, or analyze related stocks/companies? Say it anytime!