Run a Contradiction Audit on Your Own Agent Instructions

0 comments

Apps it uses

I have 31 skills and a global CLAUDE.md telling my agents how I work. Roughly 130,000 words of rules: what resolution to generate at, what order the steps go in, when to ask before spending money. Written over months, each one correct when I wrote it.

Last month one of them cost me a day. A character skill said in prose that the neutral reference sheet came before the first in-scene image, and said in its own numbered list that the hero still came first. The agent picked one, didn't mention it, and I got a character anchor contaminated with the lighting of a scene. Everything generated from it had to be redone.

The agent didn't do anything wrong. Two of my own sentences disagreed and it had to pick.

So I ran an audit. Not a spot check: one pass over everything, looking only for instructions that can't both be obeyed.

It found 47. Nine of them expensive enough to burn credits or force a redo, eighteen that quietly produce worse work, twenty that are just confusing. Plus twelve things it flagged as "this looks like a contradiction but you might know why", which turned out to be the most useful list of the three.

The pattern underneath was the part I didn't expect. Almost every one came from the same thing: I corrected a rule in one place and the old copy stayed somewhere else. Two files written months apart were the source of most of them. I wasn't writing bad rules, I was writing good rules on top of old ones.

Step-by-step

  1. I did it in two passes with two different models. One reads and reports, one fixes. The reader never edits anything, which keeps it honest about what it found.
  2. The audit pass is read-only. No edits, no commits, no generation, nothing that spends money. Its single output is one report file.
  3. I told it to do the whole thing itself with no subagents. This matters more than it sounds: a contradiction doesn't exist inside a document, it exists between two of them, so one reader holding everything beats five readers each holding a piece.
  4. I had to switch off one of my own rules to make it work. My global instructions say that when an agent hits a contradiction it stops and asks me. In an audit that halts everything at number one. So the brief says explicitly: don't stop, write it down with both versions quoted, keep going, I'll decide later on the report.
  5. Every finding has to quote both sides literally, with file and line. Not "these two disagree about resolution" but the two sentences, so I can judge without opening anything.
  6. Every finding gets a severity, and the definition is about money, not elegance. High means following the wrong branch burns credits or forces a redo. Medium means a worse result at no cost. Low means it's just confusing text.
  7. I had it check git for which version came last. Not to decide, just to tell me, because the newer sentence is usually the one I wrote to correct the older one and then forgot to delete.
  8. Then I decided the big ones myself. Six of them, all the ones that touch money. The rest went to the second model with my decisions attached.
  9. The fix pass makes one commit per contradiction. If one fix turns out wrong I can drop it without touching the other twenty-seven.
  10. Anything the fixer can't settle without me comes back as a question, all together at the end, instead of it guessing.

Result: 28 fixed, 4 partly fixed with a question attached, 15 waiting on decisions only I can make.

The thing worth stealing isn't the audit, it's the reason to run one. If you've been writing instructions for agents for a few months, you don't have a rulebook. You have sediment.

0 comments

Have a question about this workflow, or a way to improve it?

Log in to comment Create a free account

No comments yet. Ask the first question or say what worked for you.

Posting guidelines