How I Kept AI From Breaking Production

I was in the middle of moving migration data into production. Some of it was already loaded but not yet visible to users, one step short of going live. Before pushing the next batch, I ran a consistency check on what was already in, and it caught that the purchase-cost figures were off somewhere. Thousands of records, billions of won all told. Had it gone live as-is, the product amounts would have been off by that much. No human skimming the data could have caught it at that point; the verification I'd spent weeks stacking up in code did.

Only in that moment did I know this thing actually worked.

One burn, one device#

That check I opened with didn't appear one day out of nowhere. After burning four weeks in Part 1, the first thing I went looking for was a way to stop the AI from tripping over the same spot again and again. The method, such as it was, wasn't grand: every time I got burned, I drove a device into that spot. Put the AI on a task and it would fail at the very same point over and over, and each failure I answered with one more guard so it couldn't repeat the mistake. The most common failure was the AI reaching for field names or state values that didn't exist, even APIs that weren't real, and using them with full confidence. It happened so often that I wrote a hard rule into the rules document: a name you haven't verified, you don't get to use. After that, at least that one mistake dropped off sharply. Some stretches ate a lot of tokens re-reading a whole long document every time; once I hardened the routines I used often into skills, there was less to read. Up to here it was closer to first aid, patch by patch.

The thing I actually spent a long time on was something else: keeping what the AI dug out of the code from draining away each time, and getting it written down.

The devices I bolted on#

First I reworked what the AI read, and when. At the start I'd pile every document that might be needed out front, and doing that buried the one that mattered under the rest. So I changed it to feed only what a given kind of task actually needed, when it needed it. Giving the AI context was, in itself, half the work.

Next I drew a line in code that the AI couldn't cross. Anything that touched production directly, or changed data in a way that couldn't be undone, was put out of its reach entirely: blocked once by permissions, filtered again just before execution, and spelled out in the rules document on top of that. Destructive work stayed in human hands to the end; instead of running anything, the AI could only hand me verification queries as text. I hadn't handed it over out of trust. I'd fenced it because I didn't trust it.

What gave me the most trouble, though, was making the AI not forget what it had worked hard to learn. Traps you only meet after opening the code and wandering a good while were lodged all over the place:

• A field written as CSV on save but read as JSON on load
• A state value that slips just past the normal path and skips validation
• A lookup cursor that groups columns by name, letting stray values from another user slip into the result

Even when one session worked these out the hard way, the next session, not knowing they existed at all, would dig back into the same code from scratch. Every session paid full price for the same discovery, and tokens and time drained away.

So any session that touched a file had to, before it ended, write down what it had learned and what it hadn't yet confirmed. If it didn't, the session wasn't allowed to end. It was a nuisance of a step at first, but after a few sessions had piled up I felt it. There was less need to explain the same trap twice, and because each new session started on top of the notes the last one left, the results grew noticeably more accurate. Once recording was forced like that, the system got smarter the more it worked. What it learned yesterday pushed today's work along. Compounding.

The seatbelt I fitted late#

I built all of this on top of a project already in motion, and not from knowing in advance where it would break. Each time something went wrong, I patched another layer onto the spot, late. It was like fitting a seatbelt to a car already moving.

Sometimes a device had to fix a device. For a while I had the AI auto-commit every time it saved a file, and before long the commit history was buried under tiny save-by-save entries and unreadable. So I changed it to commit once, in a bundle, at the end of a session, and wrote the reason straight into a code comment so the next me wouldn't repeat the mistake.

It worked anyway. Late-patched fence or not, the thing that caught a billions-of-won error before deploy was, in the end, that device. That day I finally watched it do its job. Only later did it hit me that I should have built it far sooner.

But patching has a limit. Keep laying layer over layer on top of something moving and at some point the fence itself becomes a patchwork of rags. So for the next project I decided to flip the approach entirely: the fence before the code, the goal before anything else, all of it set up before I began. That story — building it over from the start — I wrote up in Part 3.