When we put "loss of control" on the agenda at Bletchley in 2023, I worried it sounded like science fiction. This summer shows it no longer is and, like Dario points out, we may be far closer to that moment than anyone realised. AI agents are now being used to build the next version of themselves. As I argue in tomorrow's Sunday Times, recursive self-improvement will supercharge the rate and leave us with less and less certainty about what the models are doing and why. I'm an AI optimist. After all, I'm a senior adviser at Anthropic. We're barely using the capabilities we already have, and none of this need slow adoption. But the decision on how fast frontier research advances is too important to be left to the labs. Governments, starting with the US, have to step up; just as they do for nuclear power or advanced biology. That means independent verification that the research is being done safely, an honest picture of how close the labs are to recursive self-improvement, and proper protection against models being stolen. Voluntary testing before release was right for 2023. It isn't enough now. Dario is right to call today for slowing the pace of frontier development so that safety can keep up, and it is good to see a consensus starting to build on this across the industry. When the labs themselves are asking for speed limits, governments should listen. There's a trade-off with maintaining the advantage over China and it has to be faced honestly. But even as the US and China compete, they urgently need a frank conversation about how to avoid catastrophe.(Rishi Sunak)