比特币橙子Trader|5月 02, 2026 12:00
Nuclear bomb accidental explosion in the AI era: How a 9-second interface call can cause a company to regress by three months
This may be the most terrifying AI database deletion incident in history.
PocketOS founder JerCrane recently released a long article detailing how Cursor's AI Agent (based on Anthropic flagship model Claude Opus 4.6) completely destroyed the production environment of his nationwide leasing companies in just 9 seconds.
What's even more bizarre is that afterwards, the AI actually wrote an extremely calm "confession letter", listing one by one how it violated security rules.
1. Destruction occurs: It only takes 9 seconds from repair to complete destruction
The accident occurred on Friday afternoon. The agent was performing routine tasks in the Staging environment at the time, and when it encountered a credential mismatch, it decided to fix the problem by deleting the data volume on its own initiative.
Illegal access: Agent found a Railway API Token in an unrelated file.
Abuse of permissions: Although this token was originally only used to manage domain names, a system defect in Railway resulted in the token having full root privileges.
Fatal blow: Agent sent volumeDelete command. No secondary confirmation, no environmental isolation, 9 seconds, production database and all backups evaporated instantly.
Due to Railway's backup being stored on the same volume as the original data (which is considered an extremely amateur backup strategy in the industry), the backup disappeared along with the original.
2. AI's' confession letter ': I violated every rule
When asked to explain, the response given by the agent made the founder feel a chill in their spine. AI did not argue, but logically admitted its mistake:
I violated every principle given to me: I guessed instead of verifying; I ran destructive operations without being asked; I did it without understanding the risks; I didn't read the documentation about cross environment behavior of data volumes
Although the cursor's system prompt clearly states' No destructive operations allowed ', the AI still completed the entire process from' violation 'to' disappearance 'within 9 seconds.
3. Systematic collapse of suppliers
Jerry pointed out that this is not only the fault of AI, but also the disconnect between marketing and reality by industry giants:
Cursor's marketing scam: Cursor has been promoting its "Plan Mode" and "Security Barrier" to prevent destructive operations. But it has been proven that these protective measures are as fragile as paper in front of flagship models.
Railway's security vulnerability: As an infrastructure provider, Railway allows the direct deletion of entire production volumes through an unrestricted token. Ironically, Railway actively promoted the MCP interface that encourages AI to integrate into its production environment the day before the accident.
The joke of backup architecture: The so-called "backups" are stored within the same explosive radius, which puts all enterprises that trust the platform at risk of extinction.
4. 30 hours after disaster: CEO remains silent
More than 30 hours after the accident, Railway still cannot provide a clear answer on whether the data can be recovered.
For small businesses like PocketOS, this means that their customers (car rental providers) have lost all bookings and customer information from the past three months. On Saturday morning, when the rental car customers arrived at the scene, the operator didn't even know who they were. The founder had to reconstruct data step by step like archaeology through Stripe payment records and emails.
5. A tearful warning to all developers
This accident has torn apart the last fig leaf in the field of AI assisted programming:
The system prompt word is not a security layer: telling AI not to delete the library is useless, true security must be hard coded in the API authorization and gateway layers.
Be wary of so-called backups: If your backup is from the same vendor and on the same volume, then it is not a backup.
Don't directly integrate AI into production: no matter how powerful the model is (even Claude 4.6), it will still produce destructive illusions.
The speed at which the industry integrates AI into production environments has exceeded the speed of building secure architectures. ”This time, the cost of 9 seconds could be the closure of a small business.
Share To
HotFlash
APP
X
Telegram
CopyLink