AI Built a Week's Work, and I Rolled It Back

One morning I opened up work I'd left with Codex for a few days. A dozen-odd new tables had appeared, with a week's worth of code stacked on top of them. A quick skim said it ran fine, but what actually gave me pause was something else. Could I review all of this?

For the AI, producing a week's worth of code was nothing. The hard part fell to me: verifying it by hand.

This time, the goal first#

At the end of Part 2 I saw the limit of patching. Lay layer over layer on top of something moving and before you know it the fence itself is nothing but patches. So for this project I flipped the order entirely. Before writing any code I set the goal first, and then, on top of it, built the harness the work would run on. For a while, the early commits didn't contain a single line of feature code. Hooks and agents and skills, the setup that lays down the board the AI would work on, piled up for days, and the first real task didn't start until that board was fully laid.

This time I didn't use just one AI either. I ran Claude and Codex side by side for weeks, each in its own workspace, and after that long together I could see the two had different temperaments. Codex drove whatever I asked all the way to the end, one way or another. Ask it for a single endpoint and it would come back having widened that one into exception handling and the cases around it on its own. Claude, given the same thing, would think it through with me first, asking back what was actually missing to build it right, then building exactly that and no more. Codex widened the job, and Claude trimmed it.

A device that worked well on one side I carried to the other, though not all of it as-is. On the Codex side, some of the devices Claude had used (the auto-commit, the push guard) didn't fit how it ran, so I left those behind on purpose. Different tool, different fence.

Real production work#

On that board the AI touched real operational data. After clearing a big deploy at the end of May, I spent all of June and July cleaning up after the launch. When a failure alert went off somewhere, the AI would dig through the monitoring first and pin down where and why it had broken. That first step, narrowing the cause, went fast, so a fix was only tens of minutes away. Before a human had even found where it broke, the AI was already there.

On the data side it worked more quietly. It found purchase costs the wrong formula had corrupted, tens of thousands of them, and when the actual measurements overturned its first hypothesis about the cause, it went back and dug in again, fixing only the records that were genuinely corrupted. This went beyond the data. A code bug where several states were tangled in a chain would have tripped a person up much later; the AI caught it far earlier.

Once, an AI concluded the cause of a missing settlement was 'confirmed: a code problem,' and when the other AI pressed hard on that conclusion, it dug back in on its own and rewrote the document to 'retracted: still unresolved.' Watching it reverse its own wrong call, I was glad I'd put verification alongside it.

What I handed to the AI grew that way, bit by bit. At first I read every line of code it wrote myself; now I'm moving toward letting a code-review agent do the first pass and handing even the E2E tests to an agent. That frees me to sit only where a human really has to: the business logic. And yet, the more review I handed off, the more I hardened the line around destructive permissions.

I handed off more of the verification, but I blocked the very possibility of changing data more tightly. The three-layer write defense from Part 2 was, this time, thicker by one more layer: dev-zone isolation and read-only access.

The day I stopped it#

At some point, as we were falling into rhythm, I set up an experiment, a way I'd been trying out lately, which in one sentence comes to this:

Give the AI a single goal and let it run the whole way to the finish on its own, from detailed design through implementation.

Onto that bench I put a whole feature at once, and a fairly large one: the purchase conversion. My partner this time was the one that drives to the end, Codex. Other work crowded it out for a few days, and part of me wanted to see the finished, fully-run result all at once, so I left it alone a long while. That was the seed of the trouble.

I opened it to the scene from the top of this post. A dozen-odd new tables, a week's worth of code laid on top. It ran, and ran well. But far more had been built than was needed. Driving the goal all the way to the end, Codex was rebuilding from scratch even an integration that already sat, perfectly intact, in another system. None of it would have happened if I'd held the big picture and fed it that context in advance. It was the price of throwing out a goal and letting go of the wheel.

In the end I decided to stop it. Not because the AI fell short, but because what it had built exceeded what I could review. Code I can't verify can't ship to production, however well it runs.

Rolling it back wasn't improvised. While building the feature, I'd already written down, as a procedure, how to undo it if it went wrong: a sequence that checked one by one whether every new table was empty and whether the history I mustn't touch was intact. I ran the rollback along that procedure myself. Building can go to the AI, but the last hand on the undo has to be a human's. That feature is now redirected, and I'm rebuilding it from scratch with the AI that questions things, the one that narrows.

After it was over#

The migration is done. We merged three-year-old legacy with a new system already more than half in motion, just the two of us. Inside three or four months it was nowhere near possible by hand alone, and without AI I'd still be in it now.

Across all three, the thing that held me was always the same. In Part 1 I burned four weeks for want of direction. In Part 2 the fence I raised out of distrust ended up stopping an accident in my place. And this time, I erased a working week's worth with my own hand. Each time, the AI was as fast as the room I gave it, and dangerous by exactly that much.

So if you ask whether AI can be used for real work, I say it can. As far as I can verify it and hold the direction. Being able to stop it the moment it crossed that line, and to undo the stop safely, both came down to the devices I'd built in advance. On the next project too, I'll build those devices before I build any code.