Build a Quarterly Claude AI Audit and Self-Improving Feedback Loop
I built a quarterly “AI Audit” of my Claude history, adapted from The Rundown’s AI Audit prompt and turned into a repeating loop. Each quarter, Claude reads every conversation I’ve had and reports patterns I can’t see myself: where my stated priorities differ from where my attention actually goes, which decisions I keep reopening, and connections across work, money, travel, and personal life. It then rewrites its own instructions for working with me based on what it found. I request my Claude data export, and the run finds the export email in Gmail, downloads it through Claude in Chrome, and unzips it on my machine through the desktop app. I never touch a zip file. The conversations are condensed and split by month, and Sonnet subagents pull structured notes from each month. An Opus subagent combines the monthly notes into a quarterly report under strict evidence rules: every finding cites at least three conversations across at least two months, states the strongest counter-evidence, and labels each claim as OBSERVED or INFERRED. When the evidence is thin, it says “insufficient evidence” instead of padding. Claude writes and maintains a Google Doc with two tabs. Reports contains one report per quarter with footnoted citations. Progress compares the quarters and includes two sections I added to the original prompt: Stop / Start / Continue / Change, which generates concrete behavior changes for the next quarter, and a Cross-domain pass, which identifies patterns spanning work, money, body, and personal life that I would not connect on my own. I push back on findings I disagree with. Those corrections go into the doc and a versioned prompt, now v1.1, so the next run does not repeat the mistake. Claude then takes the latest audit and its stored memory about me and proposes rewritten project instructions. The first rewrite removed stale facts and added rules to check memory, docs, and connected apps before saying it does not have something; move long-running threads into a maintained doc because one tracking chat had grown to 434 messages; and name the number that would settle an open decision because the audit showed I close decisions once a number exists and keep circling the ones without one. On the first non-holiday weekday after each quarter closes, a scheduled task emails me to request the export while I am at my desk. Scheduled checks later that day pick up the export and run the whole pipeline from a written runbook. I backfilled Q1–Q3 2026 to start, and Q3 alone covered 151 conversations and 2,922 messages. Step-by-step: 1. I started from The Rundown’s AI Audit prompt. 2. I requested my Claude data export and automated the process of finding the export email in Gmail, downloading it through Claude in Chrome, and unzipping it through the desktop app on my machine. 3. I condensed the conversations, split them by month, and used Sonnet subagents to extract structured notes from each month. 4. I used an Opus subagent to synthesize the monthly notes into a quarterly report. Each finding must cite at least three conversations across at least two months, include the strongest counter-evidence, and be labeled OBSERVED or INFERRED. Thin evidence is reported as “insufficient evidence.” 5. I had Claude write the results to a Google Doc with Reports and Progress tabs, including footnoted quarterly reports, cross-quarter comparisons, Stop / Start / Continue / Change recommendations, and a Cross-domain pass. 6. I challenged findings I disagreed with and stored those corrections in the doc and in the versioned prompt, now v1.1. 7. I had Claude use the latest audit and its stored memory about me to rewrite its project instructions, including rules about checking connected sources, maintaining long-running threads in a doc, and naming the number that would settle an open decision. 8. I scheduled the export request for the first non-holiday weekday after each quarter closes, then used scheduled checks and a written runbook to run the pipeline. 9. I backfilled Q1–Q3 2026, with Q3 covering 151 conversations and 2,922 messages. 10. To build your own version, add the evidence rules and OBSERVED/INFERRED labels, split the work into inexpensive monthly extraction and one strong synthesis, write the output to a living doc instead of a chat, and make the final step “rewrite your instructions for me.” That turns the audit from a report into a feedback loop.
0 comments