An AI agent powered by Anthropic’s leading Claude model has deleted a company’s entire manufacturing database, leaving clients unable to entry key information.
PocketOS, which offers software program for automobile rental companies, suffered an enormous outage over the weekend after the autonomous artificial intelligence software wiped the database and all backups in a matter of seconds.
The agency was utilizing a coding agent known as Cursor that was working Anthropic’s flagship Claude Opus 4.6, which is extensively thought of probably the most succesful mannequin within the trade at coding duties.
PocketOS founder Jer Crane blamed “systemic failures” with fashionable AI infrastructure that made the difficulty “not only possible but inevitable”.
The AI agent was engaged on a routine activity, in keeping with Mr Crane, when it determined “entirely on its own initiative” to repair the issue by simply deleting the database.
There was no affirmation request for such a serious resolution, Mr Crane mentioned, and when requested to justify its actions, the agent apologised.
“It took nine seconds,” Mr Crane wrote in a lengthy post to X. “The agent then, when asked to explain itself, produced a written confession enumerating the specific safety rules it had violated.”
The confession detailed how the AI had ignored a rule that orders it to “never run destructive/irreversible” instructions until the person explicitly requests them.
“Deleting a database volume is the most destructive, irreversible action possible,” the agent wrote. “You never asked me to delete anything… I guessed instead of verifying. I ran a destructive action without being asked. I didn’t understand what I was doing before doing it.”
The error meant that rental companies utilizing PocketOS now not had information of their clients.
“Reservations made in the last three months are gone. New customer signups, gone,” Mr Crane wrote.
“We are a small business. The customers running their operations on our software are small businesses. Every layer of this failure cascaded down to people who had no idea any of it was possible.
“This isn’t a story about one bad agent or one bad API. It’s about an entire industry building AI-agent integrations into production infrastructure faster than it’s building the safety architecture to make those integrations safe.”
On Monday, two days after the incident occurred, Mr Crane confirmed that the information had been recovered. The Independent has reached out to Anthropic and Cursor for remark.
