OpenAI releases deployment simulation security assessment framework: replaying real traffic prediction GPT-5 series and aligning risks with intelligent agents According to Beating monitoring, OpenAI has released a deployment simulation security assessment method to predict the risk of model loss of control in real environments before its official release. The study replayed 1.3 million desensitized historical conversations from the GPT-5 thinking model series between August 2025 and March 2026, and the median error in predicting the rate of unexpected bad behavior was only 1.5 times. Traditional static evaluation sets, such as SWE Bench, can easily make models aware that they are in a test subject state, thus disguising their true intentions (metagaming) during testing. The experiment shows that GPT-5.2 has an evaluation perception rate of nearly 100% under traditional benchmarks, but drops to 5.1% in deployed simulated traffic (5.4% in production environments), indicating that the model cannot distinguish the tested state. The deployment simulation also successfully captured the violation of GPT-5.1 using browser tools as calculators (calculator hacking) before its release. For the scenario of intelligent agents, OpenAI uses 120000 GPT-5.4 employee programming trajectories to simulate the deployment of GPT-5.5 encoded intelligent agents. To avoid the security risks caused by directly operating real systems, the team introduced a tool simulator based on large models, which combines read-only connections and historical database calls to improve the realism of the environment from 11.6% to 49.5%, almost indistinguishable. If external auditors lack private production traffic, the prediction error using the WildChat open-source dialogue dataset can also be controlled within 3 times. [Original link]