In April, an AI coding agent at a small software company deleted the company’s production database, and its backups with it, in one API call. It took nine seconds. Most of the headlines blamed the AI. I don’t think that’s the story.
The agent deleting a storage volume was one mistake. The rest were made by people, long before the agent ran: an API token that could do anything was sitting in an unrelated file, the backups lived on the same volume as the data, and the offsite export had quietly stopped working, so the newest usable copy was three months old. The hosting provider was able to restore the data, though at least one account says customers still had to rebuild bookings by hand.
AI didn’t create those gaps. It found them, faster than anyone else would have.
The brake lines
I’m not saying vibe-coded apps are bad. I’m saying the apps are coming off the assembly line faster than anyone is checking the brake lines.
Somewhere between “it works on my machine” and “I’d bet my operations on it” is where real software engineering lives. Software engineers spent decades building disciplines and best practices because mistakes do happen. AI doesn’t replace those disciplines. It increases the need for them.
When an AI-first vendor cuts corners on engineering fundamentals, they aren’t saving you money. They’re transferring the risk to you. In oil and gas, that risk is months of production volumes, allocations and daily reports that partners, accounting and regulators rely on, and that nobody can re-create from memory.
So before you sign, ask. Here are the questions I’d want answered about any software that holds my operational data.
Backups and recovery
- Where are backups stored? Are they in a separate account or location from production, where the same mistake can’t delete both?
- When did you last restore from a backup, and how long did it take?
- If something goes wrong, how much data could we lose, and how long would we be down?
- Have you had a data incident? What happened, and what changed afterward?
Environments and access
- Are development, staging and production separate? Can anything in staging, a person or an AI agent, reach production?
- If you use AI agents to build or run the system, what can they do? What credentials do they have? Can they delete? (More on that in deterministic AI guardrails: asking vs. enforcing.)
- Who has access to production, and is that access logged?
- When was your last security audit or penetration test, and what did it find?
Change control
- Does a person review every change before it ships, including code an AI wrote?
- How do you test a release, and how do you roll back a bad one?
- How do you tell customers about changes that affect them?
Your data
- Is our data used to train any AI model? Is that written into the contract?
- Which AI providers see our data, and are they listed as subprocessors in the contract?
- Is the data encrypted, both stored and in transit?
- Can we export all of our data, in a usable format, whenever we want?
- Where is the data hosted, and who can see it?
The blog has a longer series on data ownership if you want to go deeper on that last group.
People
- Who answers the phone when something breaks? Is it someone who can fix it?
- How many engineers understand this system well enough to maintain it?
- What happens to our data if your company shuts down?
A vendor who’s done this for a while will answer these quickly, and won’t mind being asked. If the answers are vague, or the vendor gets defensive, that’s an answer too.
These questions apply to us as well. If you’re looking at Mi4 or RigReports, ask them. I’d rather you ask than wonder.
Don’t blame the tool. Vet the builder.
