In addition to lacking power and computing power, AI will lack Internet data in the next step. Late at night, during my vacation, I was reading 'Situational Awareness' written by' New Stock God 'Leopold, It mentioned that AI also has a core variable: Internet data is drying up. Llama 3 training used 15 trillion tokens. But after the whole Internet is de duplicated, High quality content is just over 30 trillion yuan. So AI frontier models, I almost flipped through what humans have written. And most of these 30 trillion miles are garbage. E-commerce pages, SEO trends, advertising landing pages, Most of the GPT-4 level computing power is wasted on feeding garbage. Repeatedly feeding old data is useless, Academic research has found that by the 16th iteration, the returns have basically returned to zero. The current way of AI training, It allows the model to quickly scan through massive amounts of text, But researchers are exploring another path: allowing AI to self game and learn from failures. Synthetic data, self play, and reinforcement learning are all paths taken, Simply put, it means letting AI learn in a different way. Leopold used AlphaGo as an analogy inside. In the first stage, AlphaGo learned human chess manuals, and in the second stage, they played millions of games against themselves, Then he walked out of the 37th hand, which even Li Shishi couldn't understand, The step that is almost impossible for a human chess player to take. So for the development of language models in the 'second stage', It is currently the most critical research question. Leopold concluded by saying, If a laboratory finds a breakthrough first, That will be the key to AGI and even super intelligence, one of America's most precious secrets. I was thinking all night long, The progress of AI in the past few years has been highly transparent, For example, OpenAI publishes papers, DeepMind publishes papers, Their methodologies are mostly publicly available, Those who follow can catch up. So this round of AI competition is essentially a capital competition, It's a competition for computing power and recruitment. But if we break through the data bottleneck and the breakthrough of new AI learning methods, The competition in AI will shift from "everyone climbing together" to "winner takes all". It will be another era.