One prompt, nine production defects, and an independent AI retest

0 comments

Apps it uses

I gave Claude Fable 5.2 one goal-based prompt, an entire 58,128-entry museum, and a written list of nine independently verified production defects. Within 48 hours, a different AI agent had retested the live site: seven defects were fixed, two weren't.

I'm Ahmed Abdel Aal, a solo founder in Oakville, Canada. I built islam.museum — a free digital museum of Islamic history in Arabic, English and Urdu — alone, around a full-time job. It is a non-profit project, not a business: free for everyone, no accounts, no ads, and personally funded by me.

The workflow:

  1. Establish a baseline. A separate AI agent acceptance-tested the live museum and documented nine defects before anything changed.
  2. Give goals, not instructions. Fable 5.2 got one mission against the entire repository — verifiable statements to prove, not a step-by-step plan.
  3. Set hard gates. The museum's validator had to print "0 errors"; its citation check "0 fail."
  4. Keep deployment human. The work landed on a test branch; I reviewed, merged and deployed it myself.
  5. Retest independently. The original tester repeated the same checks against production and published the diff.

Fable's mission built the museum's Grand Tour — a River of Time with 40 cornerstone stops across 14 centuries. The same verified release also carried the new Education Explorer: 65 Ontario school courses, 360 lessons, each tied to its official curriculum codes.

The two remaining failures are public: an API file (chains.json) still returns 404, and searching "Muhammad" still ranks an object above the person's page. Fable's own run ledger also marked two of its 13 statements NOT GREEN — a different measurement from the defect list, and also published.

The engineering is documented, too: the same programme rebuilt the museum's build-and-deploy pipeline — rendering down from 3 h 06 to 43 minutes, deploy from 3 h 13 to 47 minutes — merged on main and proven on the UAT trial (not yet deployed to production; I deploy manually).

Seven independently verified fixes. Two independently verified failures. No self-certified success story.

The museum cost under CA$2,700 in cash and roughly 300 hours of my evenings and weekends. The dataset is CC0, so anyone can re-run the checks.

Links:

  • Live: https://islam.museum/en/grand-tour/
  • Repository: https://github.com/amado2k5/islam-museum
  • Engineering write-up: https://github.com/amado2k5/islam-museum/blob/main/docs/engineering/45-fable-5-2-improvements.md
  • Verification package: https://github.com/amado2k5/islam-museum/tree/main/docs/verification

0 comments

Have a question about this workflow, or a way to improve it?

Log in to comment Create a free account

No comments yet. Ask the first question or say what worked for you.

Posting guidelines