Intro to AI Evals
A beginner-friendly course on building a first AI evaluation with a spreadsheet, Claude Code, known outcomes, and a practical accuracy-and-cost scorecard.
- 6 lessons
- 1 hr 17 min
- Taught by Nate Grahek
What you'll learn
- Find Practical AI Wins Identify narrow, repeatable AI workflow opportunities where reliability, cost, speed, and accuracy can be measured.
- Design Reliable Evals Build workflow evals using real inputs, expected outputs, prompts, models, and answer keys from historical human decisions.
- Choose Automation Targets Select automation candidates with clear categories, meaningful volume, business value, and low-risk human-in-the-loop deployment paths.
- Measure Model Tradeoffs Compare prompts and models on the same task to balance output quality against latency and operating cost.
- Boost Eval Accuracy Improve accuracy with targeted business rules, examples, context, and decomposed workflow steps.
- Preserve Workflow Knowledge Capture tacit human expertise and manage AI context so workflow knowledge stays usable across experiments.
Lessons 6 lessons · 1 hr 17 min
Every lesson comes with Pro, $29 a month.
- Lesson 1: Workshop Introduction and AI ROI Mindset 12 min
- Lesson 2: Why Evals Matter for Practical AI Work 13 min
- Lesson 3: Finding Good Automation Candidates 10 min
- Lesson 4: Building the Lead Triage Eval Demo 15 min
- Lesson 5: Improving Accuracy with Prompts and Decomposition 15 min
- Lesson 6: Resources, Context Management, and Wrap-Up 12 min
Your teacher
Nate Grahek
AI Educator @ The Rundown University
Nate is a SaaS founder and Fractional CMO who helps product-driven businesses build marketing systems that actually work — without the fluff. He's spent years helping founders and operators cut through marketing complexity and put the right things on autopilot.
At Rundown University, Nate brings that same hands-on, no-jargon approach to AI education. His workshops and courses focus on practical automation and AI workflows you can deploy the same day — no engineering background required. If you've ever wanted to use AI to get your time back, Nate shows you exactly how.