What do the "last days of human dominance" look like? (This is assuming that a *permanent* global stop to AI development does not happen) First, recall my definition of AGI: AGI is AI that is powerful enough that if you were to upload it into robot bodies, and all humans suddenly disappeared, it would be able to independently continue civilization. Why this definition, and not something simpler like "AI that can do 90% of economic tasks at median human level"? Because I think this definition captures the core thing that makes AGI "not just a tool", that make it uniquely powerful and unsettling: In a "on-the-ground" near-term sense, it means that no specific single job is truly safe from automation. Some jobs might have temporary human advantage, but if they needed to, AIs could eventually figure their way around anything. But also in a more high-level megapolitical sense, once we have AGI, the fact that it's humans who are masters of the Earth, and not robots, becomes a mere historical contingency (we have the property rights and the master keys to the guns, bots don't), and not an inherent result of bots having any fundamental limitation that we do not have. Even if technology were to advance no further, over centuries and millennia as the world continues to do a random walk constantly restructuring its political organization, there is no guarantee that eventually bots would not be on top - and then win permanently due to their greater ability to mass produce themselves. So AGI, defined as I define it, is a "point of no return" that merits great caution.(vitalik.eth)