Run a Contradiction Audit on Your Own Agent Instructions
Apps it uses
I have 31 skills and a global CLAUDE.md telling my agents how I work. Roughly 130,000 words of rules: what resolution to generate at, what order the steps go in, when to ask before spending money. Written over months, each one correct when I wrote it.
Last month one of them cost me a day. A character skill said in prose that the neutral reference sheet came before the first in-scene image, and said in its own numbered list that the hero still came first. The agent picked one, didn't mention it, and I got a character anchor contaminated with the lighting of a scene. Everything generated from it had to be redone.
The agent didn't do anything wrong. Two of my own sentences disagreed and it had to pick.
So I ran an audit. Not a spot check: one pass over everything, looking only for instructions that can't both be obeyed.
It found 47. Nine of them expensive enough to burn credits or force a redo, eighteen that quietly produce worse work, twenty that are just confusing. Plus twelve things it flagged as "this looks like a contradiction but you might know why", which turned out to be the most useful list of the three.
The pattern underneath was the part I didn't expect. Almost every one came from the same thing: I corrected a rule in one place and the old copy stayed somewhere else. Two files written months apart were the source of most of them. I wasn't writing bad rules, I was writing good rules on top of old ones.
Step-by-step
- I did it in two passes with two different models. One reads and reports, one fixes. The reader never edits anything, which keeps it honest about what it found.
- The audit pass is read-only. No edits, no commits, no generation, nothing that spends money. Its single output is one report file.
- I told it to do the whole thing itself with no subagents. This matters more than it sounds: a contradiction doesn't exist inside a document, it exists between two of them, so one reader holding everything beats five readers each holding a piece.
- I had to switch off one of my own rules to make it work. My global instructions say that when an agent hits a contradiction it stops and asks me. In an audit that halts everything at number one. So the brief says explicitly: don't stop, write it down with both versions quoted, keep going, I'll decide later on the report.
- Every finding has to quote both sides literally, with file and line. Not "these two disagree about resolution" but the two sentences, so I can judge without opening anything.
- Every finding gets a severity, and the definition is about money, not elegance. High means following the wrong branch burns credits or forces a redo. Medium means a worse result at no cost. Low means it's just confusing text.
- I had it check git for which version came last. Not to decide, just to tell me, because the newer sentence is usually the one I wrote to correct the older one and then forgot to delete.
- Then I decided the big ones myself. Six of them, all the ones that touch money. The rest went to the second model with my decisions attached.
- The fix pass makes one commit per contradiction. If one fix turns out wrong I can drop it without touching the other twenty-seven.
- Anything the fixer can't settle without me comes back as a question, all together at the end, instead of it guessing.
Result: 28 fixed, 4 partly fixed with a question attached, 15 waiting on decisions only I can make.
The thing worth stealing isn't the audit, it's the reason to run one. If you've been writing instructions for agents for a few months, you don't have a rulebook. You have sediment.
- Industry
- Cross-industry
0 comments
Have a question about this workflow, or a way to improve it?
Log in to comment Create a free accountNo comments yet. Ask the first question or say what worked for you.
Posting guidelines