Workflow Hub

I’m building an AI travel assistant called SundayAtlas, and I wanted a systematic way to keep learning from what travelers are talking about between individual user conversations. Reddit is useful for this because people are unusually candid about trip-planning frustrations, destinations, bad experiences, and what they wish travel products did better. The problem is volume: I didn’t want to manually read hundreds of posts every week, so I built a Reddit Signal Agent that does the first pass and sends me a weekly travel-insights newsletter. Each week, the workflow collects posts from selected travel subreddits and passes them through an LLM-based classification and analysis pipeline. The report is organized around: - New or intensifying signals - Steady baseline themes - Fading signals - Rising destinations - Competitor mentions - Anomalies unusual enough to warrant attention This week, for example, the agent analyzed 77 posts. It surfaced a spike in discussion around short-term rental restrictions in Tokyo, growing payment friction for travelers in Japan, increased interest in quieter alternatives to heavily touristed Asian destinations, and recurring trust issues involving travel platforms. I use the newsletter as one input into product discovery for SundayAtlas. It gives me a weekly pulse on problems and behaviors that may be worth investigating further, rather than relying purely on my own assumptions about what travelers need. Step-by-step: 1. I collect recent posts from a defined set of travel subreddits. 2. I clean and structure the Reddit data for analysis. 3. I run the posts through an LLM using a defined signal taxonomy. 4. I aggregate the classifications across the weekly sample to identify patterns, changes, and anomalies. 5. I generate the report in a consistent newsletter format. 6. I run the full pipeline automatically with GitHub Actions so a new report is produced each week. I built the agent in Node.js and used Claude Code extensively during development. Evaluation ended up being the most important part. Early outputs looked convincing, but I had no objective way to know whether the classifications were actually good. I manually labeled 91 Reddit posts and created a blind golden dataset, then built a deterministic scorer to compare the agent’s classifications with my labels. The first held-out evaluation scored only 0.23, which gave me something concrete to improve against. I iterated on the classification approach and inspected individual failures. Along the way, I found three separate defects in the data collection pipeline. The held-out score eventually improved to 0.61, while the score across the full dataset increased from 0.33 to 0.67. The finished loop is: Reddit conversations → signal classification → trend analysis → weekly insights newsletter → product discovery for SundayAtlas The golden dataset remains underneath the workflow as a regression test, so when I change the agent, I can measure whether I’ve actually improved it.