HYBRID's (YC21) AI coach botched 1 in 4 chats and logged zero errors. The bug was a tool retry that kept misfiring. The agent still produced an answer every time, so analytics saw a completed session and observability saw a clean trace. The only thing that would have caught it was a conversion drop, far too late and very expensive. Almost every team shipping agents has some version of this blind spot right now. Analytics says a session happened and observability shows the trace. Neither of them inherently show whether the reply was any good, whether the user trusted it, or whether it moved them toward converting. That is the whole business case for the agent, and plenty of teams answer it with vibes and a weekly eyeball of transcripts. Amplitude hit the same gap with its own agents in 2025 and built Agent Analytics for itself. Now it's opening the tool up to other teams. As per Amplitude, every trace gets scored for task success, failure type, and tool-call errors, and every conversation becomes a measurable conversion event. The pitch is you can rank model providers on conversion, cluster conversations by intent, and fix the failing topic that costs the most first. Three customer results came out with the launch. Amplitude says The Economist runs Lens, its AI assistant, on Agent Analytics at a 96.9% task success rate with weekly failures down 84%. HYBRID fixed its 1 in 4, and the same numbers show athletes who use the AI coach retaining 4x more than athletes who skip it. Cashbook (YC21) discovered that a bad first AI answer quietly cost it retention across their app by 15%. Most agents in production have a 1 in 4 hiding somewhere. Reply quality is turning into a metric teams track on purpose, and the ones counting it now will catch theirs before the revenue chart does. No one builds AI agents just for the sake of it, especially as AI is increasingly expensive. Ultimately you're trying to drive adoption and business outcomes. I’d love to hear your thoughts. How are you measuring your agents today?