AI Doesn't Replace Engineering Discipline: Questions to Ask an AI-First Software Vendor

Two car factory scenes side by side. Left, labeled shipped fast: cars roll off the line past an empty inspector's chair and an untouched clipboard. Right, labeled shipped safe: an inspector checks the underside of a car with a flashlight, with a checklist clipped to the lift.

In April, an AI coding agent at a small software company deleted the company’s production database, and its backups with it, in one API call. It took nine seconds. Most of the headlines blamed the AI. I don’t think that’s the story.

The agent deleting a storage volume was one mistake. The rest were made by people, long before the agent ran: an API token that could do anything was sitting in an unrelated file, the backups lived on the same volume as the data, and the offsite export had quietly stopped working, so the newest usable copy was three months old. The hosting provider was able to restore the data, though at least one account says customers still had to rebuild bookings by hand.

AI didn’t create those gaps. It found them, faster than anyone else would have.

Continue reading “AI Doesn't Replace Engineering Discipline: Questions to Ask an AI-First Software Vendor”

Deterministic AI Guardrails: Asking vs. Enforcing

Two mountain roads side by side. Left, labeled deterministic hard guardrails: a steel guardrail on a concrete barrier runs along the cliff edge. Right, labeled non-deterministic soft guardrails: the same kind of road with no barrier, only a small sign that says Please stay on road.

When someone says “our AI has guardrails,” ask what kind. Most of the time they mean instructions the AI is supposed to follow. That’s a boundary the AI can fail to stay inside.

Picture an AI agent that’s told to only edit draft reports. It does, for weeks. Then a user asks it to “clean up last month’s reports so they match the new format,” and the agent decides that last month’s approved reports count. Nothing stopped it, because the only thing in its way was a sentence.

Continue reading “Deterministic AI Guardrails: Asking vs. Enforcing”
Privacy Policy Made with ❤ in Texas.