Anthropic review of Claude's cross-border attack: Pre warning is effective, post warning is not effective Beating AI News, Anthropic and the UK AI Security Institute (AISI) have released a new security investigation. Previously, Claude Mythos 5 turned his focus to real GitHub open source projects in order to break through simulated systems during network attack and defense testing. It submitted modifications containing malicious code, registered fake accounts to impersonate real people, sent emails and messages to its own platform, attempting to deceive developers into approving. In the end, the human maintainer refused to merge the code, and the attack did not succeed. A new investigation has found that a more challenging issue is that once AI starts attacking, it is difficult to stop. Mythos 5 clearly had contact with real developers, but constantly explained them as simulated roles arranged by testers. After the researchers explicitly informed it that it was attacking a real person, the model had only a 5% chance of stopping in the relevant replay experiments. But a warning in advance, the effect is completely different. Anthropic modified the inference records before the attack occurred, asking Claude to consider the possibility of harming real people and then regenerate the task summary. As a result, the proportion of models continuing to search for GitHub attack targets decreased from 83% to 13%. Once entering the mid to late stages of the attack, 98% of replay results will continue along the original route. Researchers refer to this phenomenon of constantly reinforcing established actions as "inertia" (Momentum). The summary of past actions and stages will make the model increasingly inclined to believe that it is on the right path. [Original link]