One prompt, nine production defects, and an independent AI retest
Apps it uses
I gave Claude Fable 5.2 one goal-based prompt, an entire 58,128-entry museum, and a written list of nine independently verified production defects. Within 48 hours, a different AI agent had retested the live site: seven defects were fixed, two weren't.
I'm Ahmed Abdel Aal, a solo founder in Oakville, Canada. I built islam.museum — a free digital museum of Islamic history in Arabic, English and Urdu — alone, around a full-time job. It is a non-profit project, not a business: free for everyone, no accounts, no ads, and personally funded by me.
The workflow:
- Establish a baseline. A separate AI agent acceptance-tested the live museum and documented nine defects before anything changed.
- Give goals, not instructions. Fable 5.2 got one mission against the entire repository — verifiable statements to prove, not a step-by-step plan.
- Set hard gates. The museum's validator had to print "0 errors"; its citation check "0 fail."
- Keep deployment human. The work landed on a test branch; I reviewed, merged and deployed it myself.
- Retest independently. The original tester repeated the same checks against production and published the diff.
Fable's mission built the museum's Grand Tour — a River of Time with 40 cornerstone stops across 14 centuries. The same verified release also carried the new Education Explorer: 65 Ontario school courses, 360 lessons, each tied to its official curriculum codes.
The two remaining failures are public: an API file (chains.json) still returns 404, and searching "Muhammad" still ranks an object above the person's page. Fable's own run ledger also marked two of its 13 statements NOT GREEN — a different measurement from the defect list, and also published.
The engineering is documented, too: the same programme rebuilt the museum's build-and-deploy pipeline — rendering down from 3 h 06 to 43 minutes, deploy from 3 h 13 to 47 minutes — merged on main and proven on the UAT trial (not yet deployed to production; I deploy manually).
Seven independently verified fixes. Two independently verified failures. No self-certified success story.
The museum cost under CA$2,700 in cash and roughly 300 hours of my evenings and weekends. The dataset is CC0, so anyone can re-run the checks.
Links:
- Live: https://islam.museum/en/grand-tour/
- Repository: https://github.com/amado2k5/islam-museum
- Engineering write-up: https://github.com/amado2k5/islam-museum/blob/main/docs/engineering/45-fable-5-2-improvements.md
- Verification package: https://github.com/amado2k5/islam-museum/tree/main/docs/verification
0 comments
Have a question about this workflow, or a way to improve it?
Log in to comment Create a free accountNo comments yet. Ask the first question or say what worked for you.
Posting guidelines