A memory system can top a benchmark and still not help your agent finish its tasks. The benchmarks guide points to MemoryArena, which found that "systems performing well on benchmarks such as LoCoMo can still have low task-completion rates." Recall is not the same as useful recall. The useful test is the boring one: memory on vs memory off, on your own workload. DoorDash ran that test over a 7-day production window for its Ask DoorDash assistant and reported: • ~24% higher relative checkout conversion in grocery • ~15% higher relative conversion on restaurant-assistant queries • ~33% less likely to misunderstand user intent Public benchmarks are a sanity check. The number that matters is what your agent does differently with memory than without it.