Christian 2H AGO

I built a meal tracker where my only job is to photograph a plate and send it to an agent. Identification, portion estimates, macros, storage, corrections, and the weekly review happen without me. The part worth stealing isn't the food. The agent has its own email address and iMessage thread, so anything I can send from my phone becomes an input. I send a photo through whichever channel takes fewer taps. I send corrections as plain English with `CORRECTION` in the subject, and answer questions in the same thread. There’s no app and no form. Adding a channel took an afternoon and changed what the agent could be pointed at—the tracker is just one use of an agent you can talk to. It polls on a schedule instead of using a webhook because my laptop sleeps and a local gateway would be unavailable half the time. Storage is append-only JSONL. Corrections append a new record with the same ID using last-write-wins, so the stats layer sees one meal while the original estimate stays on disk. Every number can be traced back to the model’s first guess and my override. The accuracy mechanism is a challenge step. Every photo is estimated twice: first by the main model, then by a blind subagent that receives only the photo and the rubric—not the first answer. If it identifies different food, the entry is downgraded to low confidence and becomes a question for me. Testing before making the system autonomous caught bugs that otherwise would have failed silently. The API returns attachments under a different field than I had assumed, and only on the detail endpoint. As a result, a photo email was read as having no attachments, and every meal photo would have been dropped while the daily task logged a tidy “no new meals” and appeared healthy. The send path was separately broken, which would have killed the weekly review on a Sunday even with everything upstream working. A silent no-op that produces a plausible clean run is what this category of build is prone to. The bug that actually cost me was token usage: each blind challenge read used about 44,000 tokens, so a five-photo dinner cost roughly 220,000 tokens for one meal and capped my usage. The fixes, in order of effect, were to run the challenger on a cheap model, skip it once a named product or my confirmation has settled the entry, downscale photos for viewing, and republish the dashboard only when the data changes. Measuring first mattered—I would have blamed photo size, but that was the smaller half. The honest limits are important: calorie estimates from a photo are 20–25% off at best, identification error is a bigger risk than arithmetic error, and alcohol, water, and caffeine are never estimated from photos. The sequence in which I asked for things mattered more than any single instruction: Step-by-step: 1. I gave the goal and the one ingestion mechanic I was sure about, then insisted on an agreed plan before any code. Arguing about storage and failure modes is cheap before code is attached. 2. Before scheduling anything, I processed one real input end to end in front of me, including every outbound path. Inbound gets tested because I use it; the reply and the weekly digest do not. 3. When the agent reported something about my own input that I knew was wrong, I said so and made it re-check. Confidently wrong answers about things I witnessed were the cheapest bugs to find. 4. I asked for an independent second assessment of anything estimated rather than read, and decided up front what level of agreement was enough to accept the result. 5. I asked for a visible audit surface showing every field the agent claims to track, with a correction control on it. 6. I asked the agent to look up anything knowable rather than estimate it. A named product is a lookup; only the unnameable needs a guess. 7. I treated approval friction and token cost as requirements rather than complaints, and measured before changing anything. Almost every rule exists because something went wrong in ordinary use, not because it was designed up front. An agent I can email or text, which keeps records and answers back, is general-purpose; I’ve pointed it at one narrow job. What else would you point it at?