I ship client software solo with an AI pipeline that attacks its own work—and logs every escaped bug
I never ask AI to “build the app.” I move the work through a fixed assembly line, and the most valuable stations are the ones whose only job is to attack what came out of the previous station. I’m a solo developer, and this is how I build and ship software for paying clients without a team to catch my mistakes. I choose the next chunk of work—a “phase”—and run one command. Each phase gets its own fresh context window, which matters more than any single agent because a long-running session gradually forgets its own rules. Step-by-step: 1. I discuss the phase with an agent that interrogates me until every gray area is decided. The decisions go into a file instead of staying in chat, where they can get lost. 2. A planner writes an executable plan covering the tasks, files to be changed, a threat model, and the acceptance checks that will prove the work succeeded. 3. A different agent, working from a fresh context, checks the plan by working backward from the goal and trying to prove that the plan will not achieve it. This agent can block the phase, and regularly does. 4. I execute the plan task by task, making one atomic commit for each task so the changes can be reverted cleanly. 5. I send the diff to a different model than the one that wrote it. I run Codex and CodeRabbit alongside Claude. This is the highest-value station in the line: my own tests verify only what I thought to check, while an independent model can catch the class of problem I did not anticipate. 6. An agent verifies whether the phase goal was achieved by re-deriving it from the actual code. “All tasks completed” and “the thing works” are different claims. Treating them as the same is how you ship a green checklist on top of a broken feature. 7. Before anything reaches a client, I run a security and handoff audit in a real browser against a throwaway clone of the production database. I check every page, every button, and every empty and error state. The part that compounds is what happens when a bug reaches me anyway—whether I find it in production or, worse, a client reports it. I log it as an escape, then walk the chain backward and ask each gate why it missed the problem: the planner, plan checker, executor, both reviewers, verifier, security check, and handoff audit. A one-off escape becomes a written rule. A repeat becomes a change to the gate itself. I have 42 logged escapes. That file is the most valuable thing I own because every entry represents a hole that is now closed. The pipeline I run today is mostly shaped by bugs that got past the pipeline I ran a year ago. The results so far: 29 projects, 17,657 commits, and 574 phase folders. I’ve completed six client engagements, with apps live in production and handed off to their owners, as well as mobile apps built and pushed through App Store review—all as one person, with no team. There are real costs and failure modes. A phase with every gate enabled costs meaningfully more tokens than simply asking a model for the code. That is worth it on client work, where a bug can cost me a relationship, but it is overkill for a throwaway script, so I turn the gates off for those. Gates can also be confidently wrong. My most expensive recurring failure is a check that passes on a signal adjacent to the thing it claims to verify—a green light that means nothing. My rule now is that I do not trust a new check until I have watched it fail against known-bad input. A green result you have never seen turn red is not evidence. A green test suite is not the same as a working feature. Nearly every bug that escaped me was covered by a passing test whose mock had quietly pre-satisfied the exact condition under test. Testing the real boundary is the only thing that catches those failures. This process is not hands-off. I deliberately run one phase per session, and I read what comes back. Anyone selling a fully autonomous overnight build is selling a merge conflict plus a confident summary of work that did not happen. If you want to take one idea from this, it is not the framework. The agent that writes the work must never be the one that approves it. Keep a running log of everything that gets through anyway, then fix the checkpoint that let it through instead of only fixing the bug.
0 comments