With AI agents, more user engagement does not necessarily mean more value.
A user can spend five minutes talking to an agent and leave frustrated because it never answered their question.
Instead, you have to look at the traces - the actual conversations, responses, and tool calls, to understand whether the agent did its job.
That’s why I’m excited to try
@Amplitude_HQ's new Agent Analytics. It combines traces with evals that measure the quality of agent interactions, then connects both to what users do next.
This way, teams can see:
- What the user was trying to accomplish
- Where the agent failed
- Whether good and bad answers affected KPIs
Early results show why this matters:
- The Economist (see below) reached 96.9% task success and cut weekly task failures by 84%.
- Across 20K+ Amplitude users, a positive first agent experience was associated with 3× higher retention.
Check out Agent Analytics here: