How does an AI coach that can't touch medicine prove it's safe? Numan, a UK digital health company, built Nu, an AI coach that is a wellness product: no diagnosis, no prescriptions. It had to hold that line while still coaching well. The team first scored replies one by one on static datasets, but averages hid a few dangerous replies, and scripted conversations couldn't follow Nu's unexpected turns. Worse, some low scores came from a rubric demanding empathy and advice within two turns. The turning point was treating evaluation criteria, not prompts, as the scarce asset. They test the exact same LangGraph graph patients use, score by theme, let safety probes block regressions at release, and have a cheaper model play patients so a judge can score whole conversations. A provider migration that usually takes weeks took about a day. How Numan built an eval-first AI health coach with LangGraph and LangSmith #LangGraph# #LLMEvaluation#